{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center>手写数字分类识别入门体验</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实现一个图片分类应用\n",
    "## 概述\n",
    "下面我们通过一个实际样例，带领大家体验MindSpore基础的功能，对于一般的用户而言，完成整个样例实践会持续20~30分钟。\n",
    "\n",
    "本例子会实现一个简单的图片分类的功能，整体流程如下：\n",
    "\n",
    "1. 处理需要的数据集，这里使用了MNIST数据集。\n",
    "\n",
    "2. 定义一个网络，这里我们使用LeNet网络。\n",
    "\n",
    "3. 自定义回调函数收集模型的损失值和精度值。\n",
    "\n",
    "4. 加载数据集并进行训练，训练完成后，查看结果及保存模型文件。\n",
    "\n",
    "5. 加载保存的模型，进行推理。\n",
    "\n",
    "6. 验证模型，加载测试数据集和训练后的模型，验证结果精度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "说明：<br/>你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet/lenet.py>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练的数据集下载"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 方法一：\n",
    "从以下网址下载，并将数据包解压缩后放至Jupyter的工作目录下：<br/>训练数据集：{\"<http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz>\"}\n",
    "<br/>测试数据集：{\"<http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz>\"}<br/>我们用下面代码查询jupyter的工作目录。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:31.241068Z",
     "start_time": "2020-09-04T06:46:31.232345Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Administrator'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练数据集放在----`Jupyter工作目录+\\MNIST_Data\\train\\`，此时train文件夹内应该包含两个文件，`train-images-idx3-ubyte`和`train-labels-idx1-ubyte` <br/>测试数据集放在----`Jupyter工作目录+\\MNIST_Data\\test\\`，此时test文件夹内应该包含两个文件，`t10k-images-idx3-ubyte`和`t10k-labels-idx1-ubyte`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 方法二：\n",
    "直接执行下面代码，会自动进行训练集的下载与解压，但是整个过程根据网络好坏情况会需要花费几分钟时间。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:31.263831Z",
     "start_time": "2020-09-04T06:46:31.242077Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "******Downloading the MNIST dataset******\n"
     ]
    }
   ],
   "source": [
    "import urllib.request   \n",
    "from urllib.parse import urlparse\n",
    "import gzip \n",
    "import os\n",
    "\n",
    "def unzip_file(gzip_path):\n",
    "    \"\"\"\n",
    "    unzip dataset file\n",
    "    \n",
    "    Args:\n",
    "        gzip_path (str): dataset file path\n",
    "    \"\"\"\n",
    "    open_file = open(gzip_path.replace('.gz',''), 'wb')\n",
    "    gz_file = gzip.GzipFile(gzip_path)\n",
    "    open_file.write(gz_file.read())\n",
    "    gz_file.close()\n",
    "    \n",
    "def download_dataset():\n",
    "    \"\"\"Download the dataset from http://yann.lecun.com/exdb/mnist/.\"\"\"\n",
    "    print(\"******Downloading the MNIST dataset******\")\n",
    "    train_path = \"./MNIST_Data/train/\" \n",
    "    test_path = \"./MNIST_Data/test/\"\n",
    "    train_path_check = os.path.exists(train_path)\n",
    "    test_path_check = os.path.exists(test_path)\n",
    "    if train_path_check == False and test_path_check == False:\n",
    "        os.makedirs(train_path)\n",
    "        os.makedirs(test_path)\n",
    "    train_url = {\"http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\"}\n",
    "    test_url = {\"http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\"}\n",
    "    \n",
    "    for url in train_url:\n",
    "        url_parse = urlparse(url)\n",
    "        # split the file name from url\n",
    "        file_name = os.path.join(train_path,url_parse.path.split('/')[-1])\n",
    "        if not os.path.exists(file_name.replace('.gz', '')):\n",
    "            file = urllib.request.urlretrieve(url, file_name)\n",
    "            unzip_file(file_name)\n",
    "            os.remove(file_name)\n",
    "            \n",
    "    for url in test_url:\n",
    "        url_parse = urlparse(url)\n",
    "        # split the file name from url\n",
    "        file_name = os.path.join(test_path,url_parse.path.split('/')[-1])\n",
    "        if not os.path.exists(file_name.replace('.gz', '')):\n",
    "            file = urllib.request.urlretrieve(url, file_name)\n",
    "            unzip_file(file_name)\n",
    "            os.remove(file_name)\n",
    "\n",
    "download_dataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这样就完成了数据集的下载解压缩工作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 处理MNIST数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于我们后面会采用LeNet这样的卷积神经网络对数据集进行训练，而采用LeNet在训练数据时，对数据格式是有所要求的，所以接下来的工作需要我们先查看数据集内的数据是什么样的，这样才能构造一个针对性的数据转换函数，将数据集数据转换成符合训练要求的数据形式。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "更多的LeNet网络的介绍不在此赘述，希望详细了解LeNet网络，可以查询<http://yann.lecun.com/exdb/lenet/>。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 查看原始数据集数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:32.448830Z",
     "start_time": "2020-09-04T06:46:31.265357Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The type of mnist_ds: <class 'mindspore.dataset.engine.datasets.MnistDataset'>\n",
      "Number of pictures contained in the mnist_ds： 60000\n",
      "The item of mnist_ds: dict_keys(['image', 'label'])\n",
      "Tensor of image in item: (28, 28, 1)\n",
      "The label of item: 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAANh0lEQVR4nO3df+xddX3H8edrWEpWMFIZUAFFGVvmFlfMd2iiMSxugrgE/ENiY5aamJVFWGbCEolbIllmZpapc3H+qINRFn9gpgRM2IQ1WxhuIX4hWIqoMFKltqOSyvgxKQXf++N7un358v3Ve8/90e/n+Uhu7rnn57snfd1z7vmc8/2kqpC09v3cpAuQNB6GXWqEYZcaYdilRhh2qRGGXWqEYdeikuxJ8luTrkP9MeyaiCT/muSZJE91r+9Nuqa1zrBrpJK8ZJnJV1bVid3rl8dWVKMM+zGmO73+oyS7kvx3khuTnJDkvUnuXDBvJfnFbvj6JJ9O8o/dkfSbSU5P8ldJfpLku0nOW7C530jynW763yU5Yd66fyfJvUkeT/LvSV63oMYPJtkFPL1C4DUmhv3YdBlwEfBq4HXAe49iuT8BTgEOAf8B3NN9/gfg4wvmfw9wIXAO8EvdsiR5PXAdcDnwcuBzwC1J1s9bdgvwDuBlVfVc90Xz6QXr//Mkj3VfPBes8t+gARn2Y9NfV9W+qjoIfB3YvMrlbqqqu6vqGeAm4JmquqGqngduBBYe2T9VVY902/kIcwEG+D3gc1V1V1U9X1U7mPvyeOOCGh+pqp8CVNX7q+r986Z/EHgNcAawHfh6knNWuwN09Az7sem/5g3/D3DiKpd7dN7wTxf5vHA9j8wb/gHwim74VcBV3Sn840keB86aN33hsi/SfVE8WVWHui+LbwIXr/LfoQH4W2rteBr4+SMfkpzewzrPmjf8SmBfN/wI8JGq+sgyyx7t45QF5CiX0VHwyL52fBv41SSbuwtp1/SwziuSnJlkI/Ah5k71AT4P/H6SN2TOhiTvSHLSalaa5GVJLuwuLL4kyXuAtwDf6KFmLcGwrxFV9X3gT4F/Bh4E7lx+iVX5InAb8HD3+rNuW7PM/W7/FPAT4CFWuEiY5LNJPtt9XNet68fAY8AfAJdWlW3tIxT/eIXUBo/sUiMMu9QIwy41wrBLjRhrO/vxWV8nsGGcm5Sa8gxP82wdWvR+haHCnuQi4JPAccDfVtVHl5v/BDbwhrx1mE1KWsZdtXPJaQOfxic5Dvgb4O3Aa4EtSV476PokjdYwv9nPBx6qqoer6lngy8Al/ZQlqW/DhP0MXviww95u3Ask2ZZkNsnsYQ4NsTlJwxgm7ItdBHjR7XhVtb2qZqpqZh3rF1lE0jgME/a9vPCpqDP5/6eiJE2ZYcL+LeDcJK9OcjzwbuCWfsqS1LeBm966PzV0JXOPJR4HXFdV9/dWmaReDdXOXlW3Arf2VIukEfJ2WakRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRQ/XiKrXqG/vuHWr5C1+xuadKVm+osCfZAzwJPA88V1UzfRQlqX99HNl/s6oe62E9kkbI3+xSI4YNewG3Jbk7ybbFZkiyLclsktnDHBpyc5IGNexp/Juqal+SU4Hbk3y3qu6YP0NVbQe2A7w0G2vI7Uka0FBH9qra170fAG4Czu+jKEn9GzjsSTYkOenIMPA2YHdfhUnq1zCn8acBNyU5sp4vVtU/9VKVxmbY9mIdOwYOe1U9DPx6j7VIGiGb3qRGGHapEYZdaoRhlxph2KVG+IjrGmfTmo7wyC41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiN8nn0NmOQz65PoeliD8cguNcKwS40w7FIjDLvUCMMuNcKwS40w7FIjbGc/BtiOrj6seGRPcl2SA0l2zxu3McntSR7s3k8ebZmShrWa0/jrgYsWjLsa2FlV5wI7u8+SptiKYa+qO4CDC0ZfAuzohncAl/Zcl6SeDXqB7rSq2g/QvZ+61IxJtiWZTTJ7mEMDbk7SsEZ+Nb6qtlfVTFXNrGP9qDcnaQmDhv3RJJsAuvcD/ZUkaRQGDfstwNZueCtwcz/lSBqVFdvZk3wJuAA4Jcle4MPAR4GvJHkf8EPgXaMscq2b5j7UbeNfO1YMe1VtWWLSW3uuRdIIebus1AjDLjXCsEuNMOxSIwy71AgfcR2DaW5am2Yr7Teb5o6OR3apEYZdaoRhlxph2KVGGHapEYZdaoRhlxphO3sPbEfXscAju9QIwy41wrBLjTDsUiMMu9QIwy41wrBLjbCdXcsa9plx70GYHh7ZpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhO3sa9xa/tvq/l35o7PikT3JdUkOJNk9b9w1SX6U5N7udfFoy5Q0rNWcxl8PXLTI+E9U1ebudWu/ZUnq24phr6o7gINjqEXSCA1zge7KJLu60/yTl5opybYks0lmD3NoiM1JGsagYf8McA6wGdgPfGypGatqe1XNVNXMOtYPuDlJwxoo7FX1aFU9X1U/Az4PnN9vWZL6NlDYk2ya9/GdwO6l5pU0HVZsZ0/yJeAC4JQke4EPAxck2QwUsAe4fIQ1Tj3bc5e23L7xWffxWjHsVbVlkdHXjqAWSSPk7bJSIwy71AjDLjXCsEuNMOxSI3zEtQejbkI6lpv2bF6bHh7ZpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhO3sHduDjz3H8v0Hk+CRXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRtjOfgwYZdfE3l/QDo/sUiMMu9QIwy41wrBLjTDsUiMMu9QIwy41YjVdNp8F3ACcDvwM2F5Vn0yyEbgROJu5bpsvq6qfjK5ULWWttpX7vHq/VnNkfw64qqp+BXgjcEWS1wJXAzur6lxgZ/dZ0pRaMexVtb+q7umGnwQeAM4ALgF2dLPtAC4dVZGShndUv9mTnA2cB9wFnFZV+2HuCwE4te/iJPVn1WFPciLwVeADVfXEUSy3LclsktnDHBqkRkk9WFXYk6xjLuhfqKqvdaMfTbKpm74JOLDYslW1vapmqmpmHev7qFnSAFYMe5IA1wIPVNXH5026BdjaDW8Fbu6/PEl9SVUtP0PyZuDfgPuYa3oD+BBzv9u/ArwS+CHwrqo6uNy6XpqN9Ya8ddiap85abfoaNZvW+ndX7eSJOpjFpq3Yzl5VdwKLLgysveRKa5R30EmNMOxSIwy71AjDLjXCsEuNMOxSI/xT0j0Ytr14mtvpbQtfOzyyS40w7FIjDLvUCMMuNcKwS40w7FIjDLvUCNvZp4Bt2RoHj+xSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCsEuNMOxSIwy71AjDLjVixbAnOSvJvyR5IMn9Sf6wG39Nkh8lubd7XTz6ciUNajV/vOI54KqquifJScDdSW7vpn2iqv5ydOVJ6suKYa+q/cD+bvjJJA8AZ4y6MEn9Oqrf7EnOBs4D7upGXZlkV5Lrkpy8xDLbkswmmT3MoaGKlTS4VYc9yYnAV4EPVNUTwGeAc4DNzB35P7bYclW1vapmqmpmHet7KFnSIFYV9iTrmAv6F6rqawBV9WhVPV9VPwM+D5w/ujIlDWs1V+MDXAs8UFUfnzd+07zZ3gns7r88SX1ZzdX4NwG/C9yX5Ejfwh8CtiTZDBSwB7h8JBVK6sVqrsbfCWSRSbf2X46kUfEOOqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGmHYpUYYdqkRhl1qRKpqfBtLfgz8YN6oU4DHxlbA0ZnW2qa1LrC2QfVZ26uq6hcWmzDWsL9o48lsVc1MrIBlTGtt01oXWNugxlWbp/FSIwy71IhJh337hLe/nGmtbVrrAmsb1Fhqm+hvdknjM+kju6QxMexSIyYS9iQXJflekoeSXD2JGpaSZE+S+7puqGcnXMt1SQ4k2T1v3MYktyd5sHtftI+9CdU2Fd14L9PN+ET33aS7Px/7b/YkxwHfB34b2At8C9hSVd8ZayFLSLIHmKmqid+AkeQtwFPADVX1a924vwAOVtVHuy/Kk6vqg1NS2zXAU5PuxrvrrWjT/G7GgUuB9zLBfbdMXZcxhv02iSP7+cBDVfVwVT0LfBm4ZAJ1TL2qugM4uGD0JcCObngHc/9Zxm6J2qZCVe2vqnu64SeBI92MT3TfLVPXWEwi7GcAj8z7vJfp6u+9gNuS3J1k26SLWcRpVbUf5v7zAKdOuJ6FVuzGe5wWdDM+NftukO7PhzWJsC/WldQ0tf+9qapeD7wduKI7XdXqrKob73FZpJvxqTBo9+fDmkTY9wJnzft8JrBvAnUsqqr2de8HgJuYvq6oHz3Sg273fmDC9fyfaerGe7FuxpmCfTfJ7s8nEfZvAecmeXWS44F3A7dMoI4XSbKhu3BCkg3A25i+rqhvAbZ2w1uBmydYywtMSzfeS3UzzoT33cS7P6+qsb+Ai5m7Iv+fwB9PooYl6noN8O3udf+kawO+xNxp3WHmzojeB7wc2Ak82L1vnKLa/h64D9jFXLA2Tai2NzP303AXcG/3unjS+26Zusay37xdVmqEd9BJjTDsUiMMu9QIwy41wrBLjTDsUiMMu9SI/wUQr1LnDl8n2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mindspore import context\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib\n",
    "import numpy as np\n",
    "import mindspore.dataset as ds\n",
    "\n",
    "context.set_context(mode=context.GRAPH_MODE, device_target=\"CPU\") \n",
    "train_data_path = \"./MNIST_Data/train\"\n",
    "test_data_path = \"./MNIST_Data/test\"\n",
    "mnist_ds = ds.MnistDataset(train_data_path)\n",
    "print('The type of mnist_ds:', type(mnist_ds))\n",
    "print(\"Number of pictures contained in the mnist_ds：\", mnist_ds.get_dataset_size())\n",
    "\n",
    "dic_ds = mnist_ds.create_dict_iterator()\n",
    "item = dic_ds.get_next()\n",
    "img = item[\"image\"]\n",
    "label = item[\"label\"]\n",
    "\n",
    "print(\"The item of mnist_ds:\", item.keys())\n",
    "print(\"Tensor of image in item:\", img.shape) \n",
    "print(\"The label of item:\", label)\n",
    "\n",
    "plt.imshow(np.squeeze(img))\n",
    "plt.title(\"number:%s\"% item[\"label\"])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面的运行情况我们可以看到,训练数据集`train-images-idx3-ubyte`和`train-labels-idx1-ubyte`对应的是6万张图片和6万个数字下标，载入数据后经过`create_dict_iterator`转换字典型的数据集，取其中的一个数据查看，这是一个key为`image`和`label`的字典，其中的`image`的张量(高度28，宽度28，通道1)和`label`为对应图片的数字。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集对于训练非常重要，好的数据集可以有效提高训练精度和效率，在加载数据集前，我们通常会对数据集进行一些处理。\n",
    "#### 定义数据集及数据操作\n",
    "我们定义一个函数`create_dataset`来创建数据集。在这个函数中，我们定义好需要进行的数据增强和处理操作：\n",
    "1. 定义数据集。\n",
    "2. 定义进行数据增强和处理所需要的一些参数。\n",
    "3. 根据参数，生成对应的数据增强操作。\n",
    "4. 使用`map`映射函数，将数据操作应用到数据集。\n",
    "5. 对生成的数据集进行处理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:32.459913Z",
     "start_time": "2020-09-04T06:46:32.450854Z"
    }
   },
   "outputs": [],
   "source": [
    "import mindspore.dataset.vision.c_transforms as CV\n",
    "import mindspore.dataset.transforms.c_transforms as C\n",
    "from mindspore.dataset.vision import Inter\n",
    "from mindspore.common import dtype as mstype\n",
    "\n",
    "\n",
    "def create_dataset(data_path, batch_size=32, repeat_size=1,\n",
    "                   num_parallel_workers=1):\n",
    "    \"\"\" \n",
    "    create dataset for train or test\n",
    "    \n",
    "    Args:\n",
    "        data_path (str): Data path\n",
    "        batch_size (int): The number of data records in each group\n",
    "        repeat_size (int): The number of replicated data records\n",
    "        num_parallel_workers (int): The number of parallel workers\n",
    "    \"\"\"\n",
    "    # define dataset\n",
    "    mnist_ds = ds.MnistDataset(data_path)\n",
    "\n",
    "    # define some parameters needed for data enhancement and rough justification\n",
    "    resize_height, resize_width = 32, 32\n",
    "    rescale = 1.0 / 255.0\n",
    "    shift = 0.0\n",
    "    rescale_nml = 1 / 0.3081\n",
    "    shift_nml = -1 * 0.1307 / 0.3081\n",
    "\n",
    "    # according to the parameters, generate the corresponding data enhancement method\n",
    "    resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR)\n",
    "    rescale_nml_op = CV.Rescale(rescale_nml, shift_nml)\n",
    "    rescale_op = CV.Rescale(rescale, shift)\n",
    "    hwc2chw_op = CV.HWC2CHW()\n",
    "    type_cast_op = C.TypeCast(mstype.int32)\n",
    "\n",
    "    # using map to apply operations to a dataset\n",
    "    mnist_ds = mnist_ds.map(input_columns=\"label\", operations=type_cast_op, num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(input_columns=\"image\", operations=resize_op, num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(input_columns=\"image\", operations=rescale_op, num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(input_columns=\"image\", operations=rescale_nml_op, num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(input_columns=\"image\", operations=hwc2chw_op, num_parallel_workers=num_parallel_workers)\n",
    "    \n",
    "    # process the generated dataset\n",
    "    buffer_size = 10000\n",
    "    mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size)\n",
    "    mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n",
    "    mnist_ds = mnist_ds.repeat(repeat_size)\n",
    "\n",
    "    return mnist_ds\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中\n",
    "- `batch_size`：每组包含的数据个数，现设置每组包含32个数据。\n",
    "- `repeat_size`：数据集复制的数量。\n",
    "\n",
    "先进行`shuffle`、`batch`操作，再进行`repeat`操作，这样能保证1个`epoch`内数据不重复。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们查看将要进行训练的数据集内容是什么样的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，查看数据集内包含多少组数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:32.488908Z",
     "start_time": "2020-09-04T06:46:32.460923Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of groups in the dataset: 1875\n"
     ]
    }
   ],
   "source": [
    "datas = create_dataset(train_data_path)\n",
    "print('Number of groups in the dataset:', datas.get_dataset_size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其次，取出其中一组数据，查看包含的`key`，图片数据的张量，以及下标`labels`的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:32.855325Z",
     "start_time": "2020-09-04T06:46:32.489911Z"
    },
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor of image: (32, 1, 32, 32)\n",
      "labels: [2 4 1 7 2 5 7 2 4 7 2 0 9 1 5 0 8 1 8 2 7 8 9 3 7 2 3 9 2 9 3 1]\n"
     ]
    }
   ],
   "source": [
    "data = datas.create_dict_iterator().get_next()\n",
    "images = data[\"image\"] \n",
    "labels = data[\"label\"] \n",
    "print('Tensor of image:', images.shape)\n",
    "print('labels:', labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，查看`image`的图像和下标对应的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:33.459883Z",
     "start_time": "2020-09-04T06:46:32.856341Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAADsCAYAAABKZHxbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOyde3hU1dm37zWHnAgQEggQIIREAmNRAREKimD9gKrFYkVeLUpFeUNB1Cpg7QVfixY+bQFpKaJGOYggLaDS4qHgW0U5CS9FUWokSDgHIYQQcyaZWd8fe2Yyk5lJJpOZ2Tu67uvKldl79uE3a6397HV4nrWElBKFQqFQRB+T3gIUCoXi+4oywAqFQqETygArFAqFTigDrFAoFDqhDLBCoVDohDLACoVCoRPKACsUCoVOGNIACyFihRArhBAnhBBlQohPhRC36K2rKYQQvYUQ1UKItXprCYQQYoYQYr8QokYIsVpvPY0hhChv8GcXQvxFb12etLayKoTY7iyjrjQ9rLemQLSG/IeWlQFLpMWFiAU4BYwATgK3AhuEEFdJKY/rKawJngf+V28RTVAIzAfGAPE6a2kUKWWi67MQog1wDtionyK/tMayOkNK+YreIpqileQ/tKQMSCkb/QOOA7OAz4FS4G9AHHA/sLPBsRK4wvl5NbAceA8oB3YBXYA/ASXAV8CApu7vce3PgTuNqhW4G9gAzAPWGj1d0Yzw6iDTXvcyAPwCKACEkXUavawC24Epwf4Wo6RrMPlvFK3BlgEpZdBdEBOAHwO9gKudPybY8+YCHYEaYA9wwLm9CXjOdaAQYrkQYrm/iwghOgPZwH+MqFUI0Q54GpgZ5L1009oC9Nb6C2CNdJZuo+o0ell18owQ4oIQYpcQYmQz7tka8l93rc0pA8Ea4KVSykIp5UVgC9A/yPPeklL+W0pZDbwFVEsp10gp7WhvpgGuA6WU06WU0xteQAhhBdYBr0opvzKo1t8DK6SUp4K8l55aQ0XPMpCO1rx71eA6W0NZ/TWQCXQDcoEtQogsg2oFmp3/emttVhkI1gB/4/G5EkgMdGADznl8rvKz3eh1hBAm4DXgMjAjyHtGVasQoj/wf4AlQd7HE13SNUT01DoJrfl4LIhjVVlt5DpSyr1SyjIpZY2U8lW0pvatRtTqQXPyH1pRGWjJIFwFkOBx8y4tuJYPQggBrAA6A7dKKWtbcLlIah0JZAAnNckkAmYhxJVSyoEhXC+i6RpmoqV1EvBsC85XZTUwEhAhntta8h8MWgZa4oZ2EPiBEKK/ECIObfApnLwA2ICxUsqqFl4rklpzgSy0Zk5/4EXgHTQvg1CIaLoKISzO65rRXhRxQohQX8SRLgMIIYahNZdbMvqtyioghEgSQoxx5bkQYiJwI7DVaFpdhCn/waBlIGQDLKXMRxt4+h/gCLAz1GsBCCFeFEK86PzcE5iKZtC+8fADnGg0rVLKSinlN64/tBHUaillkdG0OpmL1px6ErjX+XmuQbWCNvjyppSyLNTrqrLqTlMrmvdLEXABeBgYJ6UMyRe4teQ/GLcMiOAGFRUKhUIRbgwZCadQKBTfB5QBVigUCp1QBlihUCh0QhlghUKh0AllgBUKhUIngvL/HGW6y1CuEu87NgZ0HG8tWluLTlBaW4LK//DzXdEKqgasUCgUuqEMsEKhUOhExCdkl9f358gDvrfpscVE/OZ9kb69QhGQwlnDKLddDvi97dmL2I8URFGRIlocXzCUy118p2uI+cZKxpw9UdMRMQNsH6nNQ3NqWh3HhvtOvn9j9zuoYjCAMsQhYOnRnYIH0wHIXHGSulOndVbUujg/YxiT7t/K7OSjAY8ZtWYypiNRFNUErmfqzMi4oI6PuwCpy3ZHUlKrw5SQwMlH+/OX/3qZ0Qm+BnhbpZVHa/874PntCiRJa8JnoFUXhEKhUOhERGrA8vr+nJpWB8Dh4Wv8HvPxVW8x6uGxAFyuGkTM1v2RkNIiKn82hOr2vu+ouFIHCW/u1UGRhiUzg4JJaeTlaBPyj/pwMiZVA24Wd+Z80GjtF+D0TfFknsqgruB4dEQ1gvczFdwCJxvK27Ow4ucAdHr7a+xFIc0P9Z1CtE1k09RF2GIS/H4/OqHW/Vz5Y+bZgewrHxK25z/sBtjUry+Fs2s4PHh9k8e+b9sCwMiZ40KfEC+CjJn3EXM7+k5qP/9CX3a8GVwzMBIU3di10UKiCJ5tlVb2VWoLQmTHnWVCYqn7u7yc5diYTuaKOl27eJrzTHkyIbGUCQteAGBUwWRMHykDTF0dz50bxYK0baSa2zT79MVdDzB/XmXYnv+wG+Cq56o52G9zuC8bXYTA3LEjsSY1ANNczEntITY2tJMdEvuFCxCFGfpOVKVQ7qjm0demk/6U1k/69r33cN2CRfSy1i98kJeznOy4aWQt0Ayzo6xFsyKGhL9nqkbWUlAbeM7vtiYH3S31v+NyOyvxsbHImpqI6QyEsFgwpSQ3fWAU8t9efJGTQyD382v9Vq6ijVGXpdcVc8eO5Ozewy0JJWhTqCqC5czqNP7af0VI5x6uTSV32NCoNJXP3BrL+I13eu1L2vgp95c+zkcv5Xrt3zlxET9s+zgAvR/Sr+vJk6Ulffnw5sBLuZ0dfwWfzqlvJa1e9hz3zJlF+7WfREOeF44h/ViyrukWWzTz3ygYwgDn9lnHxLcnk/yT/Kjd8+Lb2fw0/XPeyP2R70ixSdDHep5Y4b+fSBGYDglVAfvXmuY8mEJdHad52IsvYnqkL5mlJ6lz7pM1NSRsz2PUPZMBuOvFreS0LyTV3Ia/3roMgP96ZRrZU/Qdr7j3+EguPNIdee5QwGO6bpAMtE/jwG+1Lohe1kQcluikrSeVPxvCfQu2BFkmopf/Ox8YxKg2QwH4+h6tknXs9tzGTokIhjDA2dY2/DT9c3YQ+X5Vc1J7zqxOY+NVr5BtbUPsL2tZE6etHpS2SDPEjkulTJ77OI//dr1Xn6BRkNF/jr6TOA59haPhvrIyTB99CsD52nZAIQCDY7WHdNVNK5n8ygO6GuHT5UnE7g9sfAHsRUV0eQOuq5nm3pf68Vn3yybSXJqkGbcbHt1LTvvCKN01eOT+Q24XsL6XbABct3ca5T1FVMdXomaAz9sruGHdLMzVggfGN+5/GVFiY/lr/xVkW7UO+NnJR1newBlfxMRwfqgkw3oBI3ZBdDxwiV7/yNHljd0YxxcM5bc9N/j9rqn831Zp5bEVs0gv+ywaUoPijdwfEfvLWi+tI+MdPDh4Z1QqCy3FXlRE8qr65ny0jO/FB4bSL0d7QSzueiBKdw0dx8E8ADok9ufyT6PbRx5xA7yuLIW5O+9AVJvps+AQjrIy3hl2lS4G2NKjO0ce6kGKufFOfpEQzzu3LQnYbBqccJT1v/lv0v/8GY7KykhIbRTHwTyuWD8Abo/6rf3SmHN7sPm/rzKL7s/s9qmR6knqst2siRtD5X3b+V2nL9379c7/29K+YOU83zVfu22vxrxdX4N3aZJmfFel79BVR7BUjRtMUX/NDDps5U16mqwrS+G1LTeRQXiCMVQghkKhUOhE2GvAZw50ZSTjvLazn9TeFnrXbi5ndiJ/0gtA8/3/PBmdUMumqYuYuXIc6FADaojeAQP+nNvXlaUAMO+tCV75X/mzIQxO+V+v87dXmVix7wayMV4wTtqi3ay2DeN3t9TXgPXO/9nJR5ntp5+yj20SPWv7AyB2Rb8rp/JnQ7jh0b2Ndjvce3wkUzp/zMh4b2uQX1vB3Z9NoVtN9PqLL48ZROzDZ8lzxiMEYuHFLN4pvArQ7FlmGOeKCLsBznzSW1wmx8N9C905b6/guXOjoS5avWqNk5eznMHHp9HBABFbLubuvAPAbXxd+AtumXf0dt09Cxoj5hsr2yq1sQB/8wcYhcPD13BN7D0ApFoGYK6oRTYxWBcOhMWCY0g/7luwJeCAW42sZWlJX/JfsvHKVBiZsd3r+w2l19JlXB72iKutR8684A4GC8SG8va8/uIYt6dUuO1Z1L0gzCnJxFl8C3GsqRZz53T3tuNSadScxk2xWrabO6cGZVRzS67l5JAKoCLCyhSRxtS2LSIh3ntng4CAjDl7ePSyNkFL3lSt5mkWElKSEMUXkTq8iBsGYvSwmEg0xXHQ1Ye5Xqu5fXhTZsSDG0wpySxZtzzgmEmNrOW9yg58eFMmt//rQ58XcLmjmhNVKUTzeQpkhxqy6P/9nNTVkZvQKOoGuNu7NSzp9i40GEV+pMNX/GTvF+7tyXMfj5rT+P4RzwPwzV547twoztxuCO88RRQ4/OyVvHPbEu99QQQEZFni+dN7q5jx84d0ae43DMRwrLfyz77veB3zSIev6L37nO7BDUtL+ta/CPwwPv9OTHdVEU0DHMgORZuwW5r8Vwbx4OCdAb/P6bCLRJNvH2yssGKLqXf5evy368l/oiuvbbkJIKJzdHYwJzj/w4K0bbz6r6vJtBrP/ay10OsfOdgWaw+bq0lpTkmm27s15HTYhasP/pp9WnM5ba7UZXwg/5VB/O3mZT41t0xrCUc+LKDGYWXrvBF+J14xCxPZ1jZIiyDabtn+AjFMj/Tlmvn31NeA0Z6pWxK03/Kv+4dGrDvCUXyRRydOZ/wr23y6INxaiwLfu7rOSmzxmYhoa4irHC5I2+bXDnky8OlpdHnn64h2i7TYAJv69aXquWr39qqslT4d7N4ENwA2IbEUEktZ0eWGFiqsx/rlaQYsmM6WX//RK07ek1RzG6eLVOsywEYKzog7a/GdyNxi4fHOb3tNgPJtsfa5yyF9+n97pl9wB1h4Eiusbje5kjkJ7EwcGm1pjeIvEMNx6Cs6L+xPn0cmAfWzELp+y7Y2N0bM5UnW1SF2fcZrc8bycoPZAxMLa4nZv98dADWh/Su0dBC8Rfgphw05XVfO2D88Qdc3jkS85dBiA1yXEs/2fn8Nh5aIYy8qousmQdls5X0XTmRZOeNfmoXDOQdPt4+qGz8BuPGLO+ixxVj5sPBiFst33uwV4LK46wFsmT/0OdYVWNK74FRYAxxcK8iIajN9nvwSR1kZVWu7Ysuc7j6mXYEk1s9gkNj1GaZRw7SN4WEUFSQJb+4lYMBxgwAogMknNZFVa7v6/T16oAUEPUH6qs+wB+Hh4rmyRsdd2ss8eWXwrfUWGWDTNTby7wm9pvhU0ZUArN4/LOAxrh8VLmRlFbe98xgyTmtY3D9ot5eTvaL5OCor6f5M4IEKfwEwRXu6kr7ZWKs1vFN4FbbFF+hlnQLA3370AoNjrXQaetbn2GK74IolR6k7dz6sGkqy4zl2ywuagS+bRe/nT5G0Zg9JYb2LMdj+nz4AZIdxhYlw4IiF40/0D+pYz+CjyT/QXiifpA8LejUSY1VBFAqF4ntEi2rAFwYmcez2F0I+31XzzX4wen2AjrIyrykFN80ayYejs93b3RMv8WL6PxmffydLszZ4NZmMTPEgBx0P2Nxx7UbCbwBM5Kf8bTbXppxk31XXkf2gVj62fX4Vg2O/4uOr3oq6llRzG/InvYCtejqZayzBB9kYMF3NnTpxdvwVtDXpHYrVNE2tiNEY7vDrnB3aog3LmvawaFENWLQgs7dVWon5RvvTk7RFu4kdfdz9d+GR7jx25mZMd1WxofRav+ekWr9FXt8fYdHPXc1cUcvCi1nUSK35U3DHSxwf10E3PYEwd+pEsc34E9eA1t9734ItOEYMwDFiAKnWb/0epwXijIpIIE7DZyovZzkFk9LcmhwjBmDq19fnPFO/vjhGDKC6ax3VXY0RIOSi9srufDpnudfAt27Pv3NFjPN2Y/jwR92CHKstp1qaeGzFLDIa6TfUC7n/ECeHAFS4V01INHkbkJz2hQxft5yZQ8ZhD3MfYHN0fnhzFj/Z+4WX+57RuHDbFe45aY1GSWU8JfZKtxsiaHmbs35Vo+dFMhDHVCc5VlvusyoHOfXH/Pir2zDfk+p1nmNpGe/39R0Mt0sHR+uqEHXGqho/+tp/k/FU9J//aKyI0ZzAkqgb4PtnPE6bfcdJL/tM97khmsK1akJDB3fFd4Nu9xcyaNlDHP1R4wY3mgRalcOTTdlvcGqv99PTw2LCX1DB0boqfnXLZEz5h4zYO/GdpDmBJVExwLmlaWz8pTZ9XsKBPOw6rKsVCvbii9TUZegtI2j+fN/LPByjjeBHMnClpdhyp5O58mTU5qcNhP1SKX2eKqVXuVa9bGx+ZVfQSOqSOMwVtUBkghoCrcrhSaIpDltM09fKLU1j05TRmvHVad4S14oYRqOlK2Icqy3n/hmPE/OtbzizpbgKe5CBJS0ywJ0+Pst1c6Y1eVxcqYOEj7SBDaPXehsiFndk1BNjfSbt6GEx4VhvxfRIXxyH9F/cD7QBBJdPolFoGCBiy51O5ppCXVcZ9sR+pIC+L2gOzNftDVyWO+dXASB2fRrxmqTnqhyu4Iake083OXGMJzPPDmTfgutI2LVX15pvdXuTzwsk862p9NlcoqstCLQiRrCYayVJ2z71O19Nc35XiwxwXcFxkg00A1ckiNm6n6r4wdw4TZvdyzUinmiK459932FUymTdfPkaBkAA9PhM73qlRtW4wQAk3ettaLt/WKXbtJmBcHmOJB/UWYgfXMENVSWDsfWf3uTxLtoVSJLeNGYrKGW/yVDeOqHmfzhebGrWmSCI37yPKjSD0mvsFK/vbIUXozqFnidNBUDoiWuVgabmWlUER/zmfaRvbvo4RetCBWIoFAqFTqgacJDEb94HQHaDWohetV/Do4bcFYomUQZYERHiirX/8y94Bw2YK2qVbf4eEVfq8CkDcaWtbSg+cigDrIgIrolIfMMxI79EjsI4JLy5lx1vepeBBHznV/6+ImQElypRKBQKRWDUIJxCoVDohDLACoVCoRPKACsUCoVOKAOsUCgUOqEMsEKhUOiE4Q2wEKK3EKJaCLFWby2BEEKUN/izCyH+oreuhgghYoUQK4QQJ4QQZUKIT4UQt+itKxBCiGQhxFtCiAqn5p/rrckfQgibEOIDIUSpEOJrIcQdemsKhBBihhBivxCiRgixWm89jSGE2O589l3P1WG9NQUi1LJqeAMMPA/8r94iGkNKmej6AzoDVcBGnWX5wwKcAkYA7YH/C2wQQmToqKkxngcuo6XpROAFIcQP9JXkjRDCAvwdeBtIRps6fa0QIrvRE/WjEJgPrNRbSJDM8Hi++ugtphFCK6tSykb/gOPALOBzoBT4G9rMz/cDOxscK4ErnJ9XA8uB94ByYBfQBfgTUAJ8BQxo4t53AxuAecBaI2v1uO4vgAKcPtZG1uq83ufAnUbTiraA3GUg22Pfa8CzBtPZz3mO8Ni3Dfi90dK0wTXnA6uDLCO6aAW2A1OCLcutqay6/oKtAU8Afgz0Aq52/phgz5sLdARqgD3AAef2JuA514FCiOVCiOUe2+2Ap4GZQd5LN60N+AWwRjpzwchahRCdgWzgPwbUmg3YpZT5Htc6CDRVq4i2zgYzHrv39QvynnqW1eagl9ZnhBAXhBC7hBAjDao11LIatAFeKqUslFJeBLYA/YM87y0p5b+llNXAW0C1lHKNlNKO9mYa4DpQSjldSuk54envgRVSylNB3ktPrQAIIdLRmvevtgKtVmAd8KqUMpgZ5aOtNRGtBuNJKdDWYDq/As4Ds4UQViHEaLQykEDT6Jb/IaCH1l8DmUA3IBfYIoTIMqDWUMtq0Ab4G4/Plc4bBsM5j89Vfrb9XkcI0R/4P8CSIO/jSVS1NmASWjPnWJD31EWrEMKE1kS6DMwI8p7R1loOtGuwrx3Q1HpWUdUppawFxgG3Oe89E63bLJglP/Qsq80l6lqllHullGVSyhop5ato3QK3GlBrqGW1RZPxVODxlhdCdGnBtRoyEsgATgohQPvhZiHElVLKgSFcL5JaPZkEPNvCa0RUq9ASdAXaYMGtTgMSKpHUmg9YhBC9pZRHnPuuIbjukoZENE2llJ+j1Xpd199N8K2ghkSrrIaDaGuV+O/yCQZDltWWeEEcBH4ghOgvhIhDGygLF7lAFlrToT/wIvAOMCbE60VSKwBCiGFoTaWWej9EWusLgA0YK6WsauG1IqZVSlkBvAk8LYRoI4S4HvgpWs3dMDoBhBBXCyHihBAJQohZQFe0QZ1QiLRWi/O6ZrRKTZzTkyMUIqZVCJEkhBjj0ieEmAjcCGw1mtaWlNWQDbCzw/lp4H+AI8DOUK8FIIR4UQjxovPalVLKb1x/aFX8aillkdG0evAL4E0pZYuWfI6kViFET2Aq2kvtGw//yolG0+pkOhCP1se6HpgmpWx2DTgKOu8Dzjp13gyMklL6rtZoDK1z0ZrTTwL3Oj/PNaBWK5qnRhFwAXgYGCelDMkX2KhlVU1HqVAoFDrRGgIxFAqF4juJMsAKhUKhE8oAKxQKhU4oA6xQKBQ6oQywQqFQ6ERQ/n+jTHcZylXifcfGgM7YrUVra9EJSmtLUPkffr4rWkHVgBUKhUI3lAFWKBQKnQgpBNE+UpuO4czIOOIuQOqy3S0SYbrGxvE7Ori3M1ecpO5UMHOZfPc5P2MY1R2bPi7jrRIcB/MiLygIqsYNpqi/d9Ey1UD6nz/DUVnpc3zD/Adj/R4jYunRnYIH0wFjPy/B5K2/YwC6ba/GvP1ARPX5K6suopGuqgasUCgUOhFSDbgsPRaAzsMKOXGyI6ktFHFhYBJ5OfXzMI/6cDImg77Ro82dOR8wt2PT0/VmdppKyv6hdMjX5tcRuz6LtLSAlFxhofOwQu99lfGIGCtUwuUxgyhPs7q/Kx7koOAO73m4rzsxjeSDUZHbKrmc2cn9zBj5eWn4bEN9WXXhL/8B+tgm0bNWm8o3UuX51FgHx27xP1+9jekknuhBxwOXItYaC8kAJ63Zo31YA9kcd+8Xg/phb2MlpqAoqKq7pUd3AMp7hjrDnMJFwR0vwR1wzb57AEi1DMBcUYvcfyjqWtIW7YZF3vu6WizUDemHtAi6/e5r1mZsb/QawlBj2cam2BZHly87YS8Kaa6qiNDYs+0qq01xePgaepVPASB7V1jlBYXrxdHrHznYFmcCYD9SENZ7tGQ+4HqEwNyxIzet3sPs5KPYXppO+lNNG+CCB7Q+rIZvSD0RFgumlOTAB9TUYL/UcPL7yHGiKoVyRzWJpjj3vnJHNUX2OnpZfeeHPjh4vfZhPcy/0JcdV8f5HBNthMWCKTuTpa8/T7a1jd5yAiJiYzEltW/8oCjnfzAc+O0LDK6eRofVxjHAoTzbdungaF0VdllvtEW1OezaAhHouTp2ey5ZiZMB6DOjfVjzPywG2NyxIzm793BLQgnaLHKtF8eQfixZF7jQ3P3Zg3QZF70H8MytsYzfeCf/7PuOe9/4/DupWtKNj17KjZqOluAY0o+lrz9PliVebymNcumuAaya/1yjx0Q7/79PHK2r4le3TIbiS+59fSq/BMARhfs39lztH/E8ACNXhzf/w1MDNgn6WM8TK7QJ5/9838s8HKM1HTLm7An6Msdqy7l/xuMkHMiLSoIDHF8wlPvGfujeTrVuwxYTeDmvDgktncO8ediLL2J6pC/XzL/HvS9triThRB6j7pns3tf5mWNNNuv1oPJnQ7hvwRa/NV93d8mSOIptcRz47QvRlgdoniYAv3lkXaN5D9HP/9bI8QVD+ct/vdzoMb3+kcMV670XYxF1ElP+IWRdXSTleWF79iK9anMA6Jl5noTteYyYmsPqZc951YQ7mLVysXHAK0x8ezLJP8n3e73m0mIDbOrXF8fSMnpY6h0qRifUcrlL81e6qZYm2uw7jr2sRXOaB03Bs0OZd8cGJrYt9vu9LXc6t479hMVdI+sK0xSOQ1/ReWF/j21tQML00afufafLM6Itq1EuTdIGWW54dC857Qt9vu+zYxI9l2plRuz6lJT69Q4BkFEaFiicNYxJ92uLLExI1Go2p+vKGfuHJ7BUah3RSfee5n3blugICpFopVdTuJ6p0QmBn//Mt6bSN/dbvwNb0e76P/XTLky/Qcv/6+KPMXnJA/R56CD3zJmF3Sq4ccZer+c/29qGn6Z/zg7C07XXIgMsr+9P4ewaDvZ9BzwE3fjFHfTY0riH28UHhvLDW75oye1DwtS2LYefvRIZZ2f+Db7Gd1ullUfX/jcA1V3qGJh4AoDJJ4cDULW2K7EeA4/RoqlR4Kq1XZk8fTir0ndESVFgLj4wlH452uBfw5dXr3/kEPeNhR7bqxG7Ar/Yku49TVXJYOI374uYzsJZwxh/33ZmJx9179tWaeWxFU+QvkrzWS6cNYwbOh5t5CoKT7oNPNvoMwXQx0A+3uW2y175/+DgneyoiaP92k8A2GkfyuTpbSL2XLXIAJdkx3Nw8Gqf/UV7upK+OXBwxqVJ2gOqh7EQCfG8c9sSv03NdWUpPL1pAhnzNO3DP692F6bt/+kDQPaa4LtUvo8EytsSeyWDPnoI2+ILQY0kv2/bgq3/dNI3R0oppIwu5HedvnRvu/P/md040LomJt2/1esBVQTm/Ixh/DzNe8m2hs8URKc/NxguTRrKyB807iWUtGYP20cMAo/yPDjhKOt/898BA4uagwrEUCgUCp0IqQZsusYGaA7UDZl5diDtChrvyYm/96xPDSm/toK7P5tCtxrf/sJIsvBiFu8UXgXAmQNdyWzGoKFRqPzZEG54dK/ufdWBdOTXVnDXp1PoM6PArwuPJTODgpu8PSSCKUfh5uUTw8mYswcRG8uluwbwm0fWufuF15WlAM4yokMXVFMUD3LQ8YBN16b9nTkf+LQWXGlqRBraoe1VJlbsu4Fs9nsdl5gXw8IhWQDMTj7K6IRaNk1dxMyV46CFNeCQDPDxcVrctr/olR1/GUKHEJrpG0qvpcu4POyhCGoOdXU8d24UPeO1roU3cn/knsvCiA9WMIyZ91FQ0XJ66MivrWDiF5MbzduiG7v6+IuGWo5aQvfES5wbMYDL7aysX7DIaxR87k4tciD7SWMak4I7XsJ2fjrpBoke3FejDcKdONnRK1jLyMw7ejvZU/b77E9btJuVbcYAMHtqeGMWwuOG5kFtG4G5c+PByXEW3xHSWFMt5s7pXvtkWXmL+1gaYtxeEDQAACAASURBVC++yMkhcNI5aJhKyyYS0hVnAEysKbzROeHUkXN4YthcdsJNSWU8JfZKt4vR2oztsH6781vfIBcjc7quHNNlvVXUc88uzbXLNuckdE4Fh8R+4QKoVdi9CLsB3vLrP1I2u/GuZc1lzduN45EOX/GTvd5eEeNfmkX3Z1qxgYwwRgmAMYqO5tLt/kIGLXuIoz9apbeUFjP2D07PDb2FOHEFLnyzV9s+XJtK7rChhgqXNgIhGeDMlScBsInpPk3H7pbQag6xwootxvvhdcSEdKkWY05Jptu7NeR02AUYM3RWDOrHTas1oxcrfI3evcdHcuGR7kAU5oJoEIgD9UEWaXNlQKPgGQChB/ZLpfR5qpRRL0/2+/1dL27168NsRKwVMuytxZbgalV0cEYSZ1pLOPJhATUOrax6dv19nwnJALsm2kk80SOsYgyDxcLjnd8m1awZ3z47JtF7ZfSic4LB3sbqHPDwNr6j8sYCUPfHzsTs9+3PihbfFmtp1+WQfw3+AiBc2HKnk/lxIdFIcfuRAkxH/H93vrYd0DoMsBECMbbOG0HJnAS/g8Gxwuo1QBf7y1reuf0q9/aJkx399r9GA9czIxZ3hCj3V7eoC6LjgUvYcqe7t/9878uNRsA0GwMUKgBTXiJil/5v66pxgwFtCr12KRU+39/4xR3wQicA4rdGLoDBE3PvTA7/rj1dPOZMCSYQp6EDPMB5ewU3rJtFbwNPMK4ITMKbe9mZOBRb5g+99ld3qePY7d7zK8xOPuqV//t613L38zPo8+SXOKIUCevi6+OdAcje2sgLIEK2qEUG2HEwz2vU9WHrFL8hyO1SKupn6fLgqaIrAVi9f5jf6/f4zFi1Tr24NGko32YKOg09C8Cxq97SWVE9tWntnX2o9d0PwQTi+HOAL7YLrlhylLpz5yMhVREFktbsIanBPnPvTHpZp7i359/wlk+03OBYK+/ctoSZT4+DKBvgkT84DMCXk4bWT7XrQdW4we5nL9yoQAyFQqHQibB6QfhzuLZkZlAwKQ0Ge+9feDGLTa+NBCB7kf7N+0bRwXPGFQzgb0KQQHx81VuMeljrz7rYYShxpQ4S3twbaam++Emvpn6PXoE4wbDwYhaJeTqNCH8HsB8pIPtBzUWx8mdDONC/Z8AJsPTAFYwx89EK9pUP8XlmTo11eLU6w1lWw+6G5r6wc0b8gklpPp4SG8rb8/qLY0hTo6ABMSW1Z9X855qcHrEh7lm7FkBuaRobizUHcsuBr6PetwZg7tSJ2iu7+w1ucBFMsEY0ERYLjiH9SLVuA2DlxjGkG7ySYNQVRFz57+K+BVv8epact1fw3LnREMWpKBuyuOsBchd8w6ZzozHtDTwtZjiDxiJmgBubEX/R//s5qauNXaD1RFgskJKEuYmnquEKAl3M9e4/ADntC8lZr/m4jpiaQ/y2g8iamsgJd+mKl+5gnLPjr+DTOa4y4G18S+ya29Rdn06hyzhjzI4FYEpJZsm65c1++X1vcQbiYPIdqfLOf/+UO6qZUziak0MqAN/B5UjRMBAHtGdm5OvPe00Mb4qNXLUgYgZYETrBriDRcAWBw4u7BQwqWL3sOe6ZM8s9zV4k2TlxEcV3aw9jW5ODQFFlgz56CECbIyLiqhSRwhWI08fqO3jaWP67GJ9/J6a7qoim8YXAgThZlnj+9N4qr4qN5yBzOIm6AR749DS6vPO1euAaQVpEk2un5ZamsWnKaK8VBPo8legVVOC5ykQvayIOS3T8+lLNbUhtZCkvW+50un9YRZ9Czf/XaGuseWLLnU7mypNR8UlutTgDcUJtMVTXWYktPhNmUU1jv1SKo8a3oJqFKeDzF+4Ap6gb4JS8ahWO2ATWL08zYMF0tvz6j34jC2eeHci+BdeRsGuv13hXw6CCLl924rqaae7t1I/Pht2QNKW1Ibbc6WSuKaSu4LghX8INV3hJPCFbhU+ynoEYjkulTJ77eNADxplvTSVlf70DVmJhLdEOgHDRe2Ud1+3UnpHiQQ5txeZGOF2eRGwYVxqPmAHu9lE1oIUre5JZ0AprE1Eu3PaiIrqsqmBMuydwxPp+365AkvRm07Ny2YuKSF5V/7KLRLo3pbUhmQYPsqhLief9vn+l4VwlRiOmoMgrCCrjQIlu80DImhqvFSQaBmI0xEgrYohdn5HsXPK+4wEbtqLpjR7frkCGdUWciBlg83btLZi+3Xt/qzO+OuGorGw1ExE1R2tryv9r9t1D53xjLsJZd+o06fPqX2RGmYTHXyBGQ4yitSENA8uigQrEUCgUCp1QBjgYDOpjqYgsqUvimlwMVaFoCcoNzR8NVs2IM07QjiLCmCtqmX+hr/uzevcqIokywH74Tq2aoWgWcv8hdlztGoCLwlzKiu81QqolQhQKhUIXVB+wQqFQ6IQywAqFQqETygArFAqFTigDrFAoFDqhDLBCoVDohDLACoVCoROGNcBCiAwhxLtCiBIhxDdCiGVCCEP6LQshZggh9gshaoQQq/XW0xitJV2FELFCiBVCiBNCiDIhxKdCiFv01uUPIUR5gz+7EOIveuvyR2vJfwAhhE0I8YEQolQI8bUQ4g69NQVCCLFWCHFWCPGtECJfCDGl6bMMbICB5cB5oCvQHxgBND5VkX4UAvOBlXoLCYLWkq4W4BSavvbA/wU2CCEydNTkFyllousP6AxUARt1lhWIVpH/zpfC34G3gWQgB1grhMjWVVhgngEypJTtgNuB+UKIa5s8S0rZ6B/aRJ2zgM+BUuBvaHP13Q/sbHCsBK5wfl6NltnvAeXALqAL8CegBPgKGNDIffOAWz22FwIvGVGrxzXnA6ubOk6la/PS1ePanwN3Glkn8AugAGeQk9G0tpb8B/o5zxEe+7YBvzeaVj8a+gBngQlNHRtsDXgC8GOgF3C188cEe95coCNQA+wBDji3NwHPuQ4UQiwXQnguHvVn4G4hRIIQohtwC/BPg2oNFZWuQaarEKIzkA38x8g60QzwGul8Eg2otbXkv79ZuAWaYTaaVs99lWjG+izwblM3DNYAL5VSFkopLwJb0JouwfCWlPLfUspq4C2gWkq5RkppR3szDXAdKKWcLqX0bAp9BPwA+BY4DewHNhtUa6iodA0iXYUQVmAd8KqU8isD60xHa9K/GuQ9Vf4H1voVWlfJbCGEVQgxGi1tg1n3SJcy4NxuCwwH3kQz4o0SrAH+xuNzJU2tslfPOY/PVX62/V5HCGECtqL9iDZob6AOwB+MprWFqHRt4jpOza8Bl4EZQdxPz/yfhNbMPRbkPVX+B7iOlLIWGAfc5rz3TGAD2kvDUFo9kVLapZQ7ge7AtKaOb8kgXAUebyMhRJcWXKshyUAPYJmUskZKWQysAm4N8XqR1BpuVLrWX08AK9AGtu50PpShEK38n0Twtd9AqPx3IqX8XEo5QkqZIqUcA2QC+0K8XLRtgAXIauqglhjgg8APhBD9hRBxwLwWXMsLKeUF4BgwTQhhEUIkofWthbpgSMS0gjZi67yuGTALIeJa4Nqj0rWeFwAbMFZK2ZK1gSKtEyHEMKAbLfd+UPnvRAhxtfNZShBCzELz3Fgd4uUiplUIkSqEuFsIkSiEMAshxgD3AB80dW7IBlhKmQ88DfwPcATYGeq1AIQQLwohXvTY9TO0jvQi4Gu05cQeM6jWuWhNlCeBe52f5xpUa6tIVyFET2AqWv/dNx4+thONpNODXwBvSinLWnJtlf9eWu9DG8w6D9wMjJJSNtmvqoNWidbdcBrNY2IR8Csp5d+bvE5wg7UKhUKhCDdGDsRQKBSK7zTKACsUCoVOKAOsUCgUOqEMsEKhUOhEUK5So0x3GWqk7n3HRn9hikDr0dpadILS2hJU/oef74pWUDVghUKh0A1lgBUKhUInlAFWKBQKnTDkTPjRxj5yIABnRsb5fJfxVgmOg3nRlvSdwXSNjeN3dGjyuLgLkLpsdxQUKRT1VI0bTFH/xs1gt+3VmLcfiMj9VQ1YoVAodMJQNWARG8uluwZgt2oDhx0PXIpK7fPMCK3mm5fjO8f2dSemkRzqVCURpmF6+cNcK0na+CmyJqQQ+pCQ1/enJDsegOJBDgruaHru+vkX+rJjmW8LRFGP6RobABcGJgU8pkO+NmeR2PVZVDS1Zi6PGUTsw2fJs21p9Lg+tkn0rNWmFA53ukbNAAuLBceQfkhLYGNxuZ2V9QsW0cuqTbtpe2k66TobP2EgpxZLj+4AXM7spP1vkF7+yLtcycz3x2E/dz4qGk39+lI4u4aDg1c367xU67fI63+Iae8hZF1dZMS1Ukz9+lKXEk/+PVYAjt3+QsBjr9l3DwCpFm3ucFEndUnThmW10WOLq3Acamqe/fDgaYe6/e5r1mZsb/Kcw8PX0KtcW2Mze1d49UTEAJvatkUkxHvvTEli6evPk21t08TZkZj3vHVjTmoPsbEceagHAPmTPB9ALb1O15VT5jCRYtbeGKlmLZ3NQkJKEmbAcak0MjVhITB37AgmgWNpGQf7vtPsS+S0L2T4uuXMHBK9l4UrXZtFTQ32S6WREeQHc0oyjqVlvN/3r0Edf3Dweu2D819+bQW/umUyFF+KXP47Mackg0UzKf7Lqn9+/NVtmO9Kxl58MWLaQDO+puxMHztUYq8E4Bu79/GZViuxwhpRTRExwIefvZJ3blvitc8sJFmW+ABnKBrjzOo0/tp/hdu4aosZeDP2D0/QddPXfP2YNge0q+BnWeL503ursEvB5LmP037tJ2HXZ+7YkZzde+hjPU8PiwltDUTj40rX5nD3Zw/SZVz0DHC3d2tY0u1dQk3TaOS/i27v1vB457cBGi2rDdmU/QaPvXszJ4dETBoAjiH9WPr68z52aNBHDwHQZ+YZr/03/esos5OPRlRTWA3wxbez+Wn65/y+7TJsMcEs3RQYW+50MleeRDVGoUNCVaPpOfDpaXR94whHf3UFy8a/4vWdWZjcb3tHI90/LcIk6GM936jGY7Xl3D/jcWK+raXzM8f8Nv0yrVZu+tdR/nX/UOT+Q5HR6kFT6eqPjQNeYeLbk0n+SX6EVGmYU5Lp9m4NC9K2kWhq2ogFvE4U8t9Tq6vl1RwSTXH0jC/mZIRf3NIivGq+ttzpdP+wij6F2gvV1fJy/Z5ftP+cYF4gLSEsBtic1J4zq9PYeNUrzh/Y8mp74glJ3algln+KLMMf3stOx1CS1uzRW4qbmWcH8vGy+upCl7e/5sjMK5h3xwZGJ3iv2nO6rpyxf3gCS6Uk9eOzurzQNpS357mnf0nSNm0w8HR5ht/jYoWV2clH2dbmxoi75+S/MohVWSubfV62tQ0/Tf+cHZGu5VssPN757aANWuZbU8nqW8j7jQwoyQi9f5vSuqG8PQuf+bl7+8YZe1nc1duta0L7f7Nx8xS63V8Y0S4ez+ch86NC6gqOY294UDPTviW02ACbe2dy+Hft2X/t83RohuAbv7jD/fnjq95qqYyQufjAUH54yxcBv1/c9QC2zB8SeNw5+vy7OJ3kVfUvhKMLhjLvjg1MbFvs3rfwYhYrN43BVAPpqz7DUVkZEePryv8uZu/9k08OB+CTf16l+fiu3Y2BxjN5cPBORsY79JbRIs7bK7hh3SyWjX+FrL6F3NAxss3l5uCT/6vqfbx32ocyeXobVqXvcO/Ltrbhr/1XMDN2XMQ0xRQUMeaFJxp9HgKV50jRYgNcm9aeoz9aRcPVop8quhKA1fuH+T3PfMlCWr9zfr+LJheur/UqCEakeFsaT3W+kt91+hKA29K+4PUZY9yBC/eN/dDL+AK8U3gV6fO07yNpZvzl/+STwzmU2w+A9JXGDK54bctN9Bpf5JNunlyz7x6+LW7D/YN2u9PeCKwrS2HuzjsQ1Wb6LDjEvluzGq35Go2kNXs4ZBnK5Bxt2/X8pZglXz+WRe/nYyLS+q07dZruz5wO+DyYrrGRN60tx36US0N7NvIHhwH4clJ4W8MqEEOhUCh0ImJ+wK6ab/aD+32+uzxmEJYnzgR8a997fCSJhaGuQN48EvNiWDhE8xwIOOKpc9s5bdFuVtuG8btbtFrY7OSj1OR8wM6X23PprgFkx613H7uuLAWAMwe6kslxPeTyyXtXGbbm6yJjzh7mmSfw8sCzAY9JmytJ7gY7M7LAQDXgl08MJ/vB/YjYWEoa5L9RcLcqc3ZofcAVWh9wp7e/xl5URPLKPXzSw9k6nqodm2puQ/6kFxj13mRMURz/cQW4HM5px7HbX/J7jOv3zHy0go/NQ8MW4NTyPuCKWhZezOKRDl95+cy1S6kAwDFiAOaKWveothjUr0kH6HO/6UXMR76GOxKkLdrNyjZjAJg9temILaOQav2WqtE3egVibKu08vSmCQBkzonOoKG5opb5F/p67YsL3KpvlBpZy9KSvlp5CYO2psh8svE0cgDyuQzDNe+7J17i3IgBTQbinLdXkFtyrXs7rlSfPu8JiaVMWKC5RQ6MnUZKnhakUd3V/6hEsS2OLl92wl5UFHFt5t6Z5E1rCxDQ+HqyuOsBWHBA8+opfZyE7Xk4ykJfCLvFBljuP8SHN2fxk71fYIupN8CeDuELL2bx4c1aLfOm1Xv81jRrZC0FtVqtV9RFt8ppuqz9P11XTneLb2G2x0tMbbVMaklitwRRbea8XXuppZrbkNO+kJyXcvEMxHhsxRNkPBPdmqfcf4gdV3t7BKSyG1OC1ocm2iZCXV1QTvYFtbV8eHMW8lzkXdCCwZySTJzFuyVW7qjmRFUKUBHZmzskh2tTybSW+AQDrM3YDuu3O7cCBy7lllzrlTcJ7A2/TnBrBc0PPNEU2EPkwG+bDsw48NsXGFw9jQ6rI2+A855M5tgtuV777NLB0boq7B5uIw0DnHpZE/nopVxG3TMZ00efhnz/qIQiP9LhK36yV/M0yLRa8eemtrSkr9tIm4oPRbXVn/5nLb577LdP8Okc31rwzomL+GHbxwHo/VCECnET9HnyS24omwX4jy4a+wfn6G60hQXg5KNa7PymqYt47tyoiDvZRwJ/QRDj8+/EdFcVkTbA9gsXyB02lCMfFkQ8GKCluLRiEjjWW/lnCJGQRuJoXZU7etBFwwCncBEWA+wovsijE6cz/pVt5LQv9Pk+Vli9ascNuff4SC480l23mo+jUgtFtFb4N/up5jbIOB9vwajiKCvDXBXYkdNaId2/I9q4HNd7xtf3PQxOeBkAW0wCC9K2kft5fVN4Qvt1NHRwzy1NY9OU0ZiK9a/9NhYEUV1nJbb4TIAzw4iU2IuK+Nf9Qzm4tEdQcxZ44nqmIArp6dQKYHqkL6NSJnt9XWyLC6rmawTc5TDfe/6M3s/HAGCrnu530q5QCYsBlnV1iF2f8dqcsRye878+TtZNcbo8idgoRD41RcQc1aOArtqdjuuBospSzW2Y29FzshVff/Hzte0Quz7Te7xTw48j/qi8sQCIxR0hioObcv+hgIEr/nDprPtjZ2L2R2ccxRPHoa98XKu6fNmJ62qmATD7N68zIdF/oIXeQUMzzw5k34LrSNi116ccutziEk/0COs9w9oFkfDmXnYmDsWW+UM6DdVGl/UMsgiWqnGDAUi6V//Iu0ZpxMgm3XuaqpLBxG/eFz093yO+Pt4ZgOyt0TdqVWu7Mnn68KD81fXUGQh7UREdN1Vz+NkrybBewF8X5LZKK4+taDxIIhL02GLCdmo6AO0KJElv7sHUti2Hn72Svn8qwn6kIKL3D3sfcNKaPSQBJ+c5XUyuCvcdwo9rRvym5gU1Mu/btmDrP530zXorad1YenTnyEM9PCaTUbQE14oo9jjJJ7cv8hveu64shac3TSDjmd1RH8OI37yPbpfqV8T5dt4wt9aJG2ZgOhLZ+6tADIVCodCJiHhByOv747CVR+LSEaFdgVbbmXl2YMD+6ysytLDpy2MGEaND865hmubXVvDI0Qlsyn7D7fbjsJUjr4/MzP3NxRUQ8vKJ4XRPvNTkIFJ23FnevveeqK/e4YklM4OCSWnkT1qOq5965tmB/Ls4ncS8GF00AcTfe9bw4fIN8QxuqF8Rxbv26yoj896aEDW/9YbI6/tzaprW4XF4uKaz3FHN+PwJ2kTxzuNcv6d4UHjr6GE3wK4VEQ4PNl50TiBcsd37yoeQu+Abv54cLmf8kTPHwdaoygPgyAMWjg3XpprMr61g4heT6fSLCzz27s3uaQAPD1/DNbHaighppX2jtsqAP/6YNxqA1CVxfGnrCr/d3ujxExJLuW7BIu4v1aastH55OiqO+J4U3djVZ4R7x1+G0GH1HtJ0iipsLq4AKFM//fI/2OCGuTu1CbmymwiIiQRiUD/sbaycf6zay1adt1cwp3A0pruqsBef8bMaifZ7whU0FFYD7Jq9P5QVEYxAwpt72Vg8hpz1q/SW4oU5qT2m2Ho3uJzDE0n+ST524OQQmLN3NEu6/YtEU5w7AObHS6OzykAgGq7MEAwu53aAAQum02VVRVRc61xBI7VtWrEbjBNXuo98bhyxo6N7b9eKGId/1945oY1/XKu3iOooTTnmhxtW7m/gmaPxaunVnLm9DVjA3Dk14Gok4QoaCqsBbuns/Qr/nFmdxv5rn6fhDE3u72+NZfzGO70c4KO1ykCk2PLrPzKm3RN0j0JknytoZOu0P6KWxAod14oY2lSOgSe6d63e0qdSm1/DKMFD4B00BkR8hZewGuCe8cWNhiEqQqNDQhUdzAnuBRfT5kqvQmsvvkhNXYbXOdFaZSAUXCsRfO1u1vnWlrpbElny4Ms8nKgthpgRwT5CR0z9PT0Z+PQ0urzzte+E3QovglkRw3NFlK5fHol691KwNBU0BuENGtJtWfo+OybR7p9aZmkznx3XS0qr4QnbNgDm3TuBzCd1FuOB41Ipk+c+zuO/XR/Qyd6FLXc6mWu0lQj6XtIGNjLtUym4w7evcHRCLZe7RHZWvMJZw3jgLu9OfVdAQNc3jGEoxOKOjHpirOEmBQJnX/PSMpZ0e9fv0kmu1VvMtdK9IooRXmhb542gZE5C0EFjrgCXS2u7E1fq8BusEQpRN8C9/pFD3DcWemyvxrzdOMv8uIgpKMKWOz3g9+0KJLE6vSxck4c3NoWii4UXs3gj90ekEvkmvKypof3aT3gmaSK/69j4sZkrTrqjihwH8wDok2vDVuQ/zXt8FjmX/MJZwxh/33afuRbKHCa6bvraEMYXIGbrfqriB3PjNG3QyijBTfL6/hTOrnGO+Xi3tFwrYny5vB/JzkFuI3lWewaNBUMnZzlM3hxemxVWA/zalptY0eWGRo+xLb4Q8eiSllB36jTp84wVEVe8LY1ep6a4txPzYnxG5f0esyy6M6OlBnE/f+bUcTCP9IPh19MUKaMLfVa62FdTy93vPubunzQK8Zv3UYUWsdlr7JQmjvZfRsJNSXY8Bwev9to3Km8sXx/vTMddWjM+2UBrKTbEFTSmJyoQQ6FQKHQirDXgYAZKjND/09pIW9R0zTKYYxRNs63sKno/tNdQI/MuXPN8ZBsk3DyxsJaRh7wX0RSLOxpqHgqjo9sgnEKhNydOdmR+F+/VPP5+8mqSyddJUesiZut+P0FJx3VQ0npRBljxvSV7yn52NBg8UsZXEU2ElEYam1QoFIrvD2oQTqFQKHRCGWCFQqHQCWWAFQqFQieUAVYoFAqdUAZYoVAodMKwBlgIUd7gzy6E+IveuhoihIgVQqwQQpwQQpQJIT4VQtyit65ACCHWCiHOCiG+FULkCyGajmvVCSGETQjxgRCiVAjxtRDiDr01NaQV5r/h09RFKyurM4QQ+4UQNUKI1cGeZ1gDLKVMdP0BnYEqYKPOsvxhAU4BI4D2wP8FNgghMnTU1BjPABlSynbA7cB8IcS1OmvyQQhhAf4OvA0kAznAWiFEtq7CfGk1+d+K0tRFqyirTgqB+cDKZp0lpWz0Dy20ZRbwOVAK/A1t6qP7gZ0NjpXAFc7Pq4HlwHtAObAL6AL8CSgBvgIGNHV/57V+ARTg9Fs2slbn9T4H7jS6VqAPcBaYYDStQD/nOcJj3zbg90bS2ZryP5Q0NUq6GrmsNrjmfGB1sGUl2BrwBODHQC/gauePCfa8uUBHoAbYAxxwbm8CnnMdKIRYLoRY7u8iaAZ4jXT+QiNrFUJ0BrKB/xhVq3NfJVqhOgu8a0Ct/tYHEmhGxEg6vQUaO/9DTVM9tHruM3pZDZlgDfBSKWWhlPIisAXoH+R5b0kp/y2lrAbeAqqllGuklHa0N9MA14FSyulSSp9JYYUQ6WjNu1dbgVYrsA54VUoZzIqIumh1brcFhgNvohU2o2n9CjgPzBZCWIUQo9HKQeC1bvTR6aYV5H+oaaqHVvc+jF9WQyZYA/yNx+dKgl8465zH5yo/28FcZxJa0+FYkPfURasQwgS8BlwGZgR5T93SVUppl1LuBLoD04K4Z1S1SilrgXHAbc57zwQ2AE1N1qzyP/xpGnWtnhi9rLaElkzGU4HHm1MI0aXlcvwyCXi2hdeIqFYhhABWoA0W3uos6KESrXR1YQGyQjw3olqllJ+j1dBc199N8C0hT1T+OwljmoIqqy2mJV4QB4EfCCH6CyHigHnhkVSPEGIY0I2Wez9EWusLgA0YK6WsauG1IqZVCJEqhLhbCJEohDALIcYA9wAfGE0rgBDiaiFEnBAiQQgxC+iKNljSXFT+OwljmoIqq26EEBbndc2A2ZnGTVZwQzbAUsp84Gngf4AjwM5QrwUghHhRCPFig92/AN6UUpa15NqR1CqE6AlMRetn+kbU+y1PNJpWtFHfaWhNzhJgEfArKeXfDagV4D60gZfzwM3AKCllMH2AUdPZyvIfwpSmUdDa2srqXLRuiieBe52f5zZ5neAcCxQKhUIRbgwbiKFQKBTfdZQBVigUCp1QBlihUCh0QhlghUKh0AllgBUKhUInggrEGGW6y1CuEu879+VZ3QAAEK1JREFUNvqLaQdaj9bWohOU1pag8j/8fFe0gqoBKxQKhW4oA6xQKBQ60ZK5IBSK7zTHFwzlcpdaemwxEb95n95yFN9BlAE2OOdnDKO6Y3DHdttejXn7gcgK+o5RNW4wRf39PwZ/+a+XGZ1Qi+3UdNI3R1mY4nuB6oJQKBQKnVA1YA8smRmcH9HVa1/qR2epKziujyDgzpwPmNsxmLm9oY9tEu16DQUgsbCWmK37IyntO8GpsQ6O3dLEwgaGGldvHZiusQFwYWBS0OeYayVJGz9F1oQ0N1DEkNf3pyQ73mtfuLRGzABbenQH4HJmJwBEncS09xCyri5St2wWYlA/7G2sXvsKboonL8f7YbTlTqf7hx289pkrapH7D0VcY3M5PHyNtm4AcO/xkVwo7mc4nf7SHfRJU1O/vrRLqWjyuOqudZh7ZwJgP1IQaVkBafhMNSSmoIi6U8HMrR55jo/Tnpm8qcGv2pN3uZKZ74/Dfu58pGSFxJEHLBy75QWvfeHSGpIBNiVo8xqLtoEniD/yUA8A8idpwvNrK/jVLZOh+JLvwTU12C+VhiKlWQiLBVNKMgA3rd7D7OSjTZ6Tl7NcWzvWg4UXs/jwpkzsFy5AlGaTO2+voNju36Uw02olVngbtbUZ25m/si87ro6LhrxGCSbd51+Ivtaq56o52K/pzt1jt+fSy6qtiJ79oD4G2NS2rc8z1ZDsNdPIWqA9R46yFs3gGnbs0sHRuirsUivDXczQwVy/ElKNrOVwbSo4jNXcMCe1xxRrj9j1QzLAJx/VlljaNHVRwGNSzK6EbANAliWeP723yp0Bntz92YN0GRd5A+wY0o8l67Q3cqbVCvjWxILhkQ5f0Xv3OXKHDcVeVBRGhYG5Yd0srlji/4Vx07+OBvUy0Ytwpfv3mcPPXsknt7uetzZ+j9k5cRE/bPs4AL0f2hslZcFxtK7KqwJ2eHE3jv5olfv7pSV96ys1BuLM6jT2X/s8wS2b13xCMsCOGO2/LSZ4UWZhItvqv+BsHPAKE9+eTPJP8kORExSVPxvCfQu2NKr53uMjOfebXl77Oj9zjLUZ2732xQorfaznwdRokEtY2PnAIEa1GUrvglPUBWju/Ov+oRxc2sNHpxEIJt0Bcjr8mxN7R3Pm1ljsxRejpM6be4+P5Ms1Ng781n8NUy/yXxnE325eRqrZ//PjItXcBhkXudpaqOSWprFpymhM+VoXZP4rg/jb9S/g+SKucVijVplpDh0Sqrxq6uHGEINw2dY2/DT9c3YQuSZodXsTOe0LOV1Xztg/PIGl0repk1hYS8xH3gNXZ54axHVpNr79sdZXeHj4mohp9IfcfwgT0FjPudx/iNPlGVFSFByXJmmDgTc8upec9oVNHp9qbsPjnd9npmVcpKUFZErnj3llkm6390v+K4NYddNKBsfWG6uZZwfy8bIh7u3Zv3mdCYmRb0GGwsyzA9m34DoSdu1FEvj37FtwHQkYq9ae/8ogVmWt9Nm/obw9zz39S5Iufdrie4RmgJtR8Vt4MYuVm8Z47Xtg/NaoN5k7HriELXc6phpIX/UZjsrKoM6L2bofHhjKsJ7BLsocfY4vGMpve27w2rfwYhZv5P6IVHZHXc/FB4bSL0cbUFvc1dsvudc/coj7xsIPf/wFq9J3RF2bJ1VruzJ5+nC3jpHxDkYarBXx4OCdjIx3ADD5pDbC+uXyfiSv2eM+Jv+xrmAwA9zto2oAdh4bQtKbezC1bcvhZ6/kbzcvcxtfz9+T9OaegNeKNv60ulh4MYvXXxxD6trdYXGOCckAd/pMq4/1em9Kk8cm5sWQvqjeCJyfMSyUW7YYx8E80g86PzfjvEuTNGPiaSz21dRy97uP0afyy/CKbCamhAROPtrfHTDgyTuFV5G6LPrGF+DC9bUBjesV62v5tqfZ73cpZsnXj2XR+/mYqIzmJ63Zw/YRg0DnF4E/XHk7OOFlQDNWh3L7AXgZX6PiCghyOaGJhHjeuW2JuyvKyL+noVaAp4quBGDTayNJC+NzpQIxFAqFQidCqgG74uKzQwjPvDPnA5/uh+1VJlbsu4FsjBU4UPmzIdzw6F6vZvT2KhOTP5xG9kN7m1WTDieugJG6BMHWaX+ku6XeHXBdWQoAZw50JZPjUdd2ecwgrsg457O/3FHN+Pw7OXdtPKMmfuLTNQFaP3D+pBcY9d5kTFHyZ03Mi2HhkCyAgN1iCy9mkZgXExU9LkTbRDZNXeSuhX3y3lWkr9SnRRMJjPp7zJ06cXb8FbQ11T/dCy9msem1kQCkLQqvZkMMws07ejvZU4xjfIXFgmNIP+5bsMU9gLSvRmviT/5wmm5aXY74BZPSPAJGvH2x/5g3GoAr1pbo8oKQMy/wvm2Lezu/toLtlb0ZlZBP1ZJuzPjjZp9BufzaCjaUXuveNlfURi34LG3Rbl6v1sYoanI+INX6rY++lRvHeHWjGRlXYImpX18ch4KLoFTUU3tldz6dsxzP5yqS+W8IA2wkhMWCKTuTpa8/73abO2+v4O53Nf/KbB39KwseSAfwidbz5ODg9QD8eOltmO9KjqpLlzklmTiLd1/0I0cnULWkGzkv5fLRS7l+z8s5PJHY0cc99kQ3Is7VV75jWRyOEUPJWb+qiTPqMackg8WCLCsPemA3HHgGt8Sa6mvueuZ/ozgkh2tTybSWECus2OMlprZtta8MEjQiYmO53M570O10XTmmy5G7p+oDboBjSD/+9N4qsiz1sd83rJtFnye/pM+T+g66NYdN2W/Q7d3oxtR3e7eGTdlv+OhYvey5qOqIJt3erWHx3s3u4KRo4RjSj8V7N7N472Ye6eBb09Uj/xvDfuECucOGsrSkL6AFjRx+9koOP3ulzsrquXTXAJ+yOvYPT5D+588idk9VA6beM+POnA9ItW5z13wHPj2NlLxqLQjCIG/pYEk0xdEzvpiTEfStbkjP+GISTd73SzTFkRjgNX/NvnsASJsrdetP98QVNNIcesYXY4tJYMmDL/NwouYVlDEn/KP6f77vZfbdmeXeTrVuazS4RY/8bxQpsRcVUePQaphGChpxPf+/eWQdvazeXXrWChnRlk1YDXD+K4Pomd54KOGE9utoGEo5L+sfzNt2u9c+sbhjVGbzKpw1jEn3bwXqB2FcwRpd3ziCvaio0SCIaJKxuQSA605O8/nuxhl7/Q5sRZOt80ZQMichaB3fFmvloMshY/T/u4J1PMl8ayp9Nvv2p5uT2nNmdRr/v71zj2nqDMP405ZS6Apyq1bkWi9QNjdlijPxgiZizILRTVlQYjQ6L1NMlMVscYlO53AZ02zTJWNDFo1hc0Y2GWZhiVeEmaHTaYYCK6gI3hgghQKlZX8cT+npaeUA5+b4fgkhOT20h9PTt9/5vud5n7QR3wJ4AZmX0xF9yirYsaVobUjRkjldIeik1q1ZZhZT7jswnm8Q9PPPawFenVTKoXUi206Z7O/AWbemKPO2pcLqnyR4EoHF1M1Y/S7pUGNL3jZE5V+FXcQ5PS44rlUCAEKusR+7nB4DSFyAtScu4fJ6bscx6/piRBbJYwbMuigJABCUwVRexJ5ci/jcJ87zzkCjwfeT8px3S8pKHRQX+Vuo6W2zYMnX72L/arbG2x05mVueS7wYyyLOWAVvRctrAT5SNAexSx5heUDTkJ/rN1MRTJOETSJoWTEdyS/2LfgcbQvFruNpiMkuG/AtMf0hBiBJfE1TSTg+HJWAHXpp56mbSsIRe5dt0FFq7KiYfdDpq39UPhpRP4mrLPCJjIB5dRRru356IwAw1BsAZRrxVHx9IiNQvTHSpeEU/zg6OhCRXYZM3Rp0G55dgE2fPYa92oyzkUxTSZL2HxS8/zaiPufu/BQKd2OJXLAuSnK+/zTN9g5MObcRcQ2tEHqSRB5DEAKBQBiG8DoCjtlejm+mzsRyDj1WPUGL9Tt7qIn6QLOwalD/jEbkR11wmhd2FqbByHEBRaHRoGXpZNjV1P1LUEY9aupGARicQcUbHW9MQ+DVB/3eCoVd70bp47GAxCNgT0J1lV6PxrTxaJ/lgLO1vchtX32MMW766f6pn+MP490Y9JjrGAkPlmgFqlZ8BXo6LasxUbBrlcuCnrdRWorWhuPrcpB1aBEg4QiYfv9dTUNSmFs8cTfVgdqJhYxt9+1AXNY9URrD866CuH0nDB8Z4lnbNUobNgffZDUOB/qE+LetoVAutULTdI/6GwGdXIopLyFCR835fVC6GAAw4b3+L3aVXg9bQgS6A9Uo2JPjXDUt6VAj8/f+e2MMlPk7z+HoibmIOBMMdQO1SGCvNkMZEICexHHO/cbsqGG0o6yytePnOy8jBMK1+OSKu7j9mGUE/IY+SzUgHs0aPaDiC1B6axOoRJSqdOq6rV3IblV54ctpCJZZPwO5oNLrcf/NcaKaG7jiKRHlob0d+x6kACIl9/BegCesqfDYVlKlj8T4sgdYoG1mFeE+IX770x/hmXGognPWmjIgAAotpQtuXEJfTAB9QdX3WLAlbxtisnm8oBQKqMLCoFGanakcY0+vAgDEZY1Ee1KMV2MDQJ1TIfsrc8WTuD3n42UY+d3z4SzzlIjiSq3NAmWPvFIcpMb754VCaHMDF1ShIXB80YZr8cWM7bnNr+LONPHqkGg6YFqIXX3GLOv0Bk/c2puA4tf3A8BTjzhTK5j6CaWa4FPLqgoLw9qycizQNoNuXF0x+yAA4P4lwE/BPg450rJ0Mgr25OB5ONbBsHLTVgSV/ElyO12Q4vMyUMac6sL+MacAiXXS4hkx3ITYcuKHudRtZclfEz0+vjvgAEv0XmuzYOWmrfB9YsPov6v5l6wpFYhTP4RG0fe6tIIg2HM3R8SeXItxBdSKuX+TVRbmBoePgiFuT9y1AYbiGsFXlwcD10SM3NZw/Lie6h+hvVIJh0xSfE17/0WsjRqu1y70fnckJHR6hzeTSOKuDZS+XmJVhifTUEZdMh5vjoCYVvhh64RzNQ3QTZeTNN6mJNhpBCpbLzXy6eqSRTExFq5jaFalLr50IsbMTGbvjNDKTkmiZ/TnGzF1O9vA4oquwQbDlRpM7Xr2fn6tDmjPUf+X1OfZFXu1GX6NBsa2SB8lHAVqKDfz35yne/4U9GYxjVf5Yw+xmpgfs4zAp9nLAACGX2pkGT0EAPWWIGhETuYetgVYe+ISSnXTYTK+5tQBnndbDaUF7q4EmnudDaSFvO2khfgODbf94wqbPRsGJOKJkVKHSO3Oo+kx1yGEg6jeDiAkX54Fggt0EkVcwgrcmnkYOqUffo0vxrzQVbxrTi3havzRj+LJmSCRT835Sz1YodMudgccgBzCYUUvwEeK5iDPMIOxTVfpi3AJetcGHS5HEPpMFLGpTBUDLXCXAlqIz3l/AY+FD8QUtw9n6CSKaNskvKJJd3ZHq0lXI77FxOuXdHCVtd9UHF2lL68JEnzx1mn2XY4UdYgYMQgEAkEiRB8BC9Epaqh4S/ggI7XBQxsTkm9QKcfNHf6I22SGvUVe4ZH/VxQXryK8NR7J+6jzH218iE5DGHw99BEZymtMuMjf84mBo60N4yXs6e3OsJ0DJghLEG1MOEz9MoB8oYmN48ZNaFJct9RJdCQEb5ApCAKBQJAIRW8vkZATCASCFJARMIFAIEgEKcAEAoEgEaQAEwgEgkSQAkwgEAgSQQowgUAgSAQpwAQCgSAR/wFy70xSbi9HmgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 32 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "count = 1\n",
    "for i in images:\n",
    "    plt.subplot(4, 8, count) \n",
    "    plt.imshow(np.squeeze(i))\n",
    "    plt.title('num:%s'%labels[count-1])\n",
    "    plt.xticks([])\n",
    "    count += 1\n",
    "    plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过上述三个查询操作，看到经过变换后的图片，数据集内分成了1875组数据，每组数据中含有32张图片，每张图片像数值为32×32，数据全部准备好后，就可以进行下一步的数据训练了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 构造神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在对手写字体识别上，通常采用卷积神经网络架构（CNN）进行学习预测，最经典的属1998年由Yann LeCun创建的LeNet5架构，<br/>其中分为：<br/>1、输入层；<br/>2、卷积层C1；<br/>3、池化层S2；<br/>4、卷积层C3；<br/>5、池化层S4；<br/>6、全连接F6；<br/>7、全连接；<br/>8、全连接OUTPUT。<br/>结构示意如下图:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LeNet5结构图"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://www.mindspore.cn/tutorial/zh-CN/master/_images/LeNet_5.jpg\" alt=\"LeNet5\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在构建LeNet5前，我们需要对全连接层以及卷积层进行初始化。\n",
    "\n",
    "`Normal`：参数初始化方法，MindSpore支持`TruncatedNormal`、`Normal`、`Uniform`等多种参数初始化方法，具体可以参考MindSpore API的`mindspore.common.initializer`模块说明。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用MindSpore定义神经网络需要继承`mindspore.nn.cell.Cell`，`Cell`是所有神经网络（`Conv2d`等）的基类。\n",
    "\n",
    "神经网络的各层需要预先在`__init__`方法中定义，然后通过定义`construct`方法来完成神经网络的前向构造，按照LeNet5的网络结构，定义网络各层如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:33.471087Z",
     "start_time": "2020-09-04T06:46:33.461899Z"
    }
   },
   "outputs": [],
   "source": [
    "import mindspore.nn as nn\n",
    "from mindspore.common.initializer import Normal\n",
    "\n",
    "class LeNet5(nn.Cell):\n",
    "    \"\"\"Lenet network structure.\"\"\"\n",
    "    # define the operator required\n",
    "    def __init__(self, num_class=10, num_channel=1):\n",
    "        super(LeNet5, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')\n",
    "        self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02))\n",
    "        self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02))\n",
    "        self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02))\n",
    "        self.relu = nn.ReLU()\n",
    "        self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n",
    "        self.flatten = nn.Flatten()\n",
    "\n",
    "    # use the preceding operators to construct networks\n",
    "    def construct(self, x):\n",
    "        x = self.max_pool2d(self.relu(self.conv1(x)))\n",
    "        x = self.max_pool2d(self.relu(self.conv2(x)))\n",
    "        x = self.flatten(x)\n",
    "        x = self.relu(self.fc1(x))\n",
    "        x = self.relu(self.fc2(x))\n",
    "        x = self.fc3(x) \n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建完成后，可以使用`print(LeNet5())`将神经网络中的各层参数全部打印出来，也可以使用`LeNet().{layer名称}`打印相应的参数信息。下面例子选择打印第一个卷积层和第一个全连接层的相应参数，使用如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:33.787344Z",
     "start_time": "2020-09-04T06:46:33.472097Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "layer conv1: Conv2d<input_channels=1, output_channels=6, kernel_size=(5, 5),stride=(1, 1),  pad_mode=valid, padding=0, dilation=(1, 1), group=1, has_bias=False,weight_init=normal, bias_init=zeros>\n",
      "****************************************\n",
      "layer fc1: Dense<in_channels=400, out_channels=120, weight=Parameter (name=fc1.weight, value=[[-0.00758117 -0.01498233  0.01308791 ...  0.03045311 -0.00079244\n",
      "  -0.01519072]\n",
      " [-0.00077699 -0.01607893 -0.00215094 ... -0.00235667 -0.01918699\n",
      "  -0.00828544]\n",
      " [-0.00105981 -0.01547002 -0.01332507 ...  0.01294748  0.00878882\n",
      "   0.01031067]\n",
      " ...\n",
      " [ 0.01414873 -0.02673322  0.01534838 ...  0.00437457 -0.01688845\n",
      "  -0.00188475]\n",
      " [ 0.01756713 -0.0201801  -0.0223504  ...  0.00682346 -0.00856738\n",
      "   0.00753205]\n",
      " [-0.01119993  0.01894077 -0.02048291 ...  0.03681218 -0.01461048\n",
      "   0.0045935 ]]), has_bias=True, bias=Parameter (name=fc1.bias, value=[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.])>\n"
     ]
    }
   ],
   "source": [
    "network = LeNet5()\n",
    "print(\"layer conv1:\", network.conv1)\n",
    "print(\"*\"*40)\n",
    "print(\"layer fc1:\", network.fc1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 自定义回调函数收集模型的损失值和精度值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "自定义一个收集每一步训练的`step`，每训练一个`step`模型对应的`loss`值，每训练25个`step`模型对应的验证精度值`acc`的类`StepLossAccInfo`，该类继承了`Callback`类，可以自定义训练过程中的处理措施，非常方便，等训练完成后，可将数据绘图查看`step`与`loss`的变化情况，step与acc的变化情况。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "参数解释如下：\n",
    "\n",
    "- `model`：函数的模型Model。\n",
    "- `eval_dataset`：验证数据集。\n",
    "- `step_loss`：收集step和loss值的字典，数据格式`{\"step\": [], \"loss_value\": []}`。\n",
    "- `steps_eval`：收集step和模型精度值的字典，数据格式为`{\"step\": [], \"acc\": []}`。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:33.795001Z",
     "start_time": "2020-09-04T06:46:33.788382Z"
    }
   },
   "outputs": [],
   "source": [
    "from mindspore.train.callback import Callback\n",
    "\n",
    "# custom callback function\n",
    "class StepLossAccInfo(Callback):\n",
    "    def __init__(self, model, eval_dataset, step_loss, steps_eval):\n",
    "        self.model = model\n",
    "        self.eval_dataset = eval_dataset\n",
    "        self.step_loss = step_loss\n",
    "        self.steps_eval = steps_eval\n",
    "        \n",
    "    def step_end(self, run_context):\n",
    "        cb_params = run_context.original_args()\n",
    "        cur_epoch = cb_params.cur_epoch_num\n",
    "        cur_step = (cur_epoch-1)*1875 + cb_params.cur_step_num\n",
    "        self.step_loss[\"loss_value\"].append(str(cb_params.net_outputs))\n",
    "        self.step_loss[\"step\"].append(str(cur_step))\n",
    "        if cur_step % 125 == 0:\n",
    "            acc = self.model.eval(self.eval_dataset, dataset_sink_mode=False)\n",
    "            self.steps_eval[\"step\"].append(cur_step)\n",
    "            self.steps_eval[\"acc\"].append(acc[\"Accuracy\"])\n",
    "            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 搭建训练网络并进行训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建完成神经网络后，就可以着手进行训练网络的构建，模型训练函数为`Model.train`，参数主要包含：\n",
    "1. 每个`epoch`需要遍历完成图片的`batch`数：`epoch_size`；\n",
    "2. 数据集`ds_train`；\n",
    "3. 回调函数`callbacks`包含`ModelCheckpoint`、`LossMonitor`和`Callback`模型检测参数；\n",
    "4. 数据下沉模式`dataset_sink_mode`，此参数默认`True`需设置成`False`，因为此功能不支持CPU模式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:33.810658Z",
     "start_time": "2020-09-04T06:46:33.796009Z"
    }
   },
   "outputs": [],
   "source": [
    "# training related modules\n",
    "from mindspore import Tensor\n",
    "from mindspore.train.serialization import load_checkpoint, load_param_into_net\n",
    "from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor\n",
    "from mindspore.train import Model\n",
    "from mindspore.nn.metrics import Accuracy\n",
    "\n",
    "def train_net(model, epoch_size, mnist_path, repeat_size, ckpoint_cb, step_loss_info):\n",
    "    \"\"\"Define the training method.\"\"\"\n",
    "    print(\"============== Starting Training ==============\")\n",
    "    # load training dataset\n",
    "    ds_train = create_dataset(os.path.join(mnist_path, \"train\"), 32, repeat_size)\n",
    "    model.train(epoch_size, ds_train, callbacks=[ckpoint_cb, LossMonitor(125), step_loss_info], dataset_sink_mode=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 定义损失函数及优化器\n",
    "\n",
    "在进行定义之前，先简单介绍损失函数及优化器的概念。\n",
    "\n",
    "损失函数：又叫目标函数，用于衡量预测值与实际值差异的程度。深度学习通过不停地迭代来缩小损失函数的值。定义一个好的损失函数，可以有效提高模型的性能。\n",
    "\n",
    "优化器：用于最小化损失函数，从而在训练过程中改进模型。\n",
    "\n",
    "定义了损失函数后，可以得到损失函数关于权重的梯度。梯度用于指示优化器优化权重的方向，以提高模型性能。\n",
    "\n",
    "MindSpore支持的损失函数有`SoftmaxCrossEntropyWithLogits`、`L1Loss`、`MSELoss`等。这里使用`SoftmaxCrossEntropyWithLogits`损失函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:57.649137Z",
     "start_time": "2020-09-04T06:46:33.811666Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "============== Starting Training ==============\n",
      "epoch: 1 step: 125, loss is 2.3096159\n",
      "epoch: 1 step: 250, loss is 2.3002408\n",
      "epoch: 1 step: 375, loss is 2.3008525\n",
      "epoch: 1 step: 500, loss is 2.3079991\n",
      "epoch: 1 step: 625, loss is 2.2878244\n",
      "epoch: 1 step: 750, loss is 2.3090718\n",
      "epoch: 1 step: 875, loss is 1.6479633\n",
      "epoch: 1 step: 1000, loss is 0.19777162\n",
      "epoch: 1 step: 1125, loss is 0.17173\n",
      "epoch: 1 step: 1250, loss is 0.22985725\n",
      "epoch: 1 step: 1375, loss is 0.16031101\n",
      "epoch: 1 step: 1500, loss is 0.12752411\n",
      "epoch: 1 step: 1625, loss is 0.03572363\n",
      "epoch: 1 step: 1750, loss is 0.15765305\n",
      "epoch: 1 step: 1875, loss is 0.20735049\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits\n",
    "\n",
    "if os.name == \"nt\":\n",
    "    # clean up old run files before in Windows\n",
    "    os.system('del/f/s/q *.ckpt *.meta')\n",
    "else:\n",
    "    # clean up old run files before in Linux\n",
    "    os.system('rm -f *.ckpt *.meta *.pb')\n",
    "\n",
    "lr = 0.01\n",
    "momentum = 0.9 \n",
    "\n",
    "# create the network\n",
    "network = LeNet5()\n",
    "\n",
    "# define the optimizer\n",
    "net_opt = nn.Momentum(network.trainable_params(), lr, momentum)\n",
    "\n",
    "# define the loss function\n",
    "net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
    "\n",
    "# define the model\n",
    "model = Model(network, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()} )\n",
    "\n",
    "epoch_size = 1\n",
    "mnist_path = \"./MNIST_Data\"\n",
    "# save the network model and parameters for subsequence fine-tuning\n",
    "config_ck = CheckpointConfig(save_checkpoint_steps=375, keep_checkpoint_max=16)\n",
    "# group layers into an object with training and evaluation features\n",
    "ckpoint_cb = ModelCheckpoint(prefix=\"checkpoint_lenet\", config=config_ck)\n",
    "# define step_loss dictionary for saving loss value and step number information\n",
    "eval_dataset = create_dataset(\"./MNIST_Data/test\")\n",
    "step_loss = {\"step\": [], \"loss_value\": []}\n",
    "steps_eval = {\"step\": [], \"acc\": []}\n",
    "# collect the steps,loss and accuracy informations\n",
    "step_loss_acc_info = StepLossAccInfo(model , eval_dataset, step_loss, steps_eval)\n",
    "\n",
    "repeat_size = 1\n",
    "train_net(model, epoch_size, mnist_path, repeat_size, ckpoint_cb, step_loss_acc_info)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练完成后，能在Jupyter的工作路径上生成多个模型文件，名称具体含义`checkpoint_{网络名称}-{第几个epoch}_{第几个step}.ckpt`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 查看损失函数随着训练步数的变化情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:57.780213Z",
     "start_time": "2020-09-04T06:46:57.649137Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEWCAYAAACEz/viAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dedxc4/3/8dcnixASQmIJISla2xeVEKmlWkpKba22VGv7fYVWlaK1RO1UKVr0aw1qbVBLbLXUvpQkRIgEQTaJ7KvI/vn9cZ3pnHvu2e85M3PP/X4+HvOYs5/PnLnv85nrus65jrk7IiLStrWrdQAiIlJ7SgYiIqJkICIiSgYiIoKSgYiIoGQgIiIoGUgFmNklZjbLzL6o8n5vNLM/VHOfhZjZBDPbp4r7621mbmYdqrXPSovi36LWcbR1SgYNotonodh+ewGnA9u4+4YJ7ucYM3s1Ps3dT3T3i5Pap7ROtfpfaO2UDKSlNgNmu/uMWgcibVtrLh3VAyWDNsDMjjez8WY2x8yGmVnPaLqZ2TVmNsPM5pvZaDPbLpq3v5l9YGYLzexzMzsjy3b3AZ4FeprZIjO7w8z2MrMpGcv995eamV1gZveb2Z3RtseYWb/Ysr3M7CEzm2lms83sejPbGrgRGBDtZ1607B1mdkmhzxnNczM70cw+NrO5ZvY3M7Msn6mnmX1lZuvGpn0zqgbraGabm9nzUWyzzOweM1snx3HPjK/JsYn29c/os35mZr/J8x2uYWZXmdnE6Lt61czWiC1ypJlNimIaHFtvFzN7w8zmmdm06HiuVsxxMbP20T5nRfH9Ol4lZWZrm9mQaLufW6gubJ8j/vZmdo6ZfRJ97yOjUmXKPjliyHu8o7+tM81sNPClmd0HbAo8Fv2t/D7XMZUM7q5XA7yACcA+WaZ/F5gF7AR0Aq4DXo7m7QeMBNYBDNga2CiaNw3YIxruBuyUY797AVNyjWfGBlwALAH2B9oDfwT+E81rD7wLXAOsCawO7B7NOwZ4NWO7dwCXFPqc0XwHHo8+66bATGBgjs/0PHB8bPxK4MZoeAvge9E+egAvA3/J8Vn/G1/msSH8EBsJnAesBnwN+BTYL0dMfwNeBDaOjtO3ohh6R5/tFmANYAdgKbB1tF5fYFegQ7TsWODUYo4LcCLwAbBJ9DfwXLR8h2j+I8BN0Xe1PvAWcEKO+H8HvAd8g/C3tgOwXhExFHO8RwG9gDXy/S/oVeAcUusA9KrQF5k7GQwBroiNrwUsj04M3wU+ik4W7TLWmwScAHQtsN//nuCyjWfGRkgGz8XmbQN8FQ0PiE4EHbLs5xjyJ4OcnzMad6LEEo3fD5yV4zP9L/B8NGzAZGDPHMseAryT47P+N77MYwP0ByZlbOts4PYs+2gHfAXskGVe7+izbRKb9hZweI54TwUejo3nPC6EpHhCbN4+0fIdgA0ISWeN2PwjgBdy7PdD4OAc80r5brId7+OK+V/QK/9L1USNrycwMTXi7ouA2cDG7v48cD3hV+d0M7vZzLpGi/6I8Ot9opm9ZGYDKhhT/KqjxcDqUdVDL2Ciu68oY5s5P2ee/a6VY1sPEqqkegJ7Ek5WrwCY2fpm9o+oWmQBcDfQvYx4NyNUr81LvYBzCCfZTN0JpaRP8mwv62czs6+b2eNm9kUU72VZ4s11XHoSEmFKfHgzoCMwLRb/TYQSQja9yoy/mOM9GWkxJYPGN5XwjwuAma0JrAd8DuDu17p7X2Bb4OuE4jzuPtzdDyb8cz9C+LVWjC+BzrH9tScU74sxGdjUsjcEFupeN+/nLIW7zwOeAX4C/Ay4z6OfnIRqLQe2d/euwM8JpYdsmhwLIH611WTgM3dfJ/bq4u77Z9nOLELV2ualfhbgBmAcsGUU7zl54s00jVBFlBKv459MKBl0j8Xf1d23zbGtyZQXfzHHO/NvQ10xl0HJoLF0NLPVY68OwL3AsWa2o5l1IvwyfNPdJ5jZzmbW38w6Ek5cS4CVZraamR1pZmu7+3JgAbCyyBg+IvzSPyDa7rmE+t5ivEU4AV1uZmtGn2G3aN50YJN442eGnJ+zyH1n295RhBLSvbHpXYBFwDwz25goeeYwCtjfzNY1sw0JVTQpbwELosbPNaIG1u3MbOfMjbj7KuA24Oqo0bm9mQ2IPmchXQjf3yIz2wr4ZRHrpNwPnGJmG0eNtmfGYppGSJhXmVlXM2sXNfZ+O8e2bgUuNrMtLdjezNYrMv5ij3fKdEIbjJRAyaCxPEmoW069LnD3fwN/AP5JONFuDhweLd+V0PA4l1DFMhv4czTvF8CEqGh+IuEXWUHuPh/4FeGf/3NCkpmSd6X0uiuBAwmNhpOi9X4azX4eGAN8YWazsqyb73OWYxiwJTDd3d+NTb+Q0Eg9H3gCeCjPNu4iNIhPIJw4h8biTX3WHYHPCL/+bwXWzrGtMwgNsMOBOcCfKO7/9wxC6WYh4bsemn/xJm6J4h4NvEP4+1pB+ofBUYTG7w8If0MPAhvl2NbVhOTyDCE5DSE0eBdSyvFO+SNwblR91ewqOMnO0qVfEZHczOz7hKuqNiu4sLQ6KhmISFZR9dX+ZtYhqqI5H3i41nFJMlQyEJGszKwz8BKwFaHa8QngFHdfUNPAJBFKBiIiomoiEREJdxK2Ot27d/fevXvXOgwRkVZl5MiRs9w9630/rTIZ9O7dmxEjRtQ6DBGRVsXMJuaap2oiERFRMhARESUDERFByUBERFAyEBERlAxERAQlAxERQcmget5+G956q9ZR1Mbrr8OqVc2nT5wIc+akx92zLyciiVMyKGTlSliypLR1Fi2C116DZ5+FZcvCtL59oX9/mDSp9BhmzoRHH02PL1wID+Xp1t0dzj8frryy+bwpU8AMHo46nxw1Cr76qnAMH38MN9yQfd7ixemT+OLFcMABYR933AFXXw277QbXXdd0nWXLoHdv2Hpr+OQTOPdc6NwZ2reHp59OLzNvHjz5ZPhMkyfDxhvDGWfA3XfD3Llh2fnz4YknYMIEGDwYbrwRZsxI72vpUlhRzpM0RdqQWj+EuZxX3759PVEff+x+551heKON3MH9s8/C+Jgx7u+/7z5zZnhdcIH72LHuI0a4n3lmWDbzdc892adDWMfd/fLL3Xfe2X233dxPP9392GPdn38+93rgfvHF7qNHu59zjvvrr7v/9Kfuw4a5n3xyepnrrnNfujTs47jj3Hv1CtP793c/9dT0cnfe6f4//+P+wx+6P/KI+wknhGX79Wu6zxdfdF9jDfcnnnA/5BD3ffcN09dd1719+/zxnn66+xFHuN9wQ/7lwP1nPyu8TKHXF1+Ez92lSxjv0iWMX3llGL/77mT/jkTqDDDCc5xXW2Wvpf369fOyuqN46SV48MHwS/vpp8OvTrNw6pC2oXPnUHpJOeYYuOUW6NAqe2YRKYmZjXT3ftnmta1qog8/hHvvhaFDQyIAJYK2Jp4IIFRldewYqrJWFvuYZ5HG07aSwaBBMHt2SACpemR3WL4cbr0Vhg8P9dFXXAEHHth03X4ZybRTsc94z2KLLeD3vy9++R13TA9fcUX5+621AQNqHUFur78Ou+8On39e60hEaqJtVROVwh0WLIC1Y88nnzYt/LKcMSOc2F5/HdZZJzTA7rRTKG0ceyz8/e/p9QYPhssug9/9LjSStmsXqiraxfLw3Llw6qkhWU2bBnfdBddcE6qwPv4Y9t03TN9wwzBtxowwvsMOYf1ly0KSmzo1JLbttw/7cA/xLVgAp5wCF18cGlo7dICxY2HXXcO2zzoLLr8cdtklXPH05puhAfy996BPn5CMttsORo4MSfLQQ+Gpp+DSS2HNNcP8oUNh4EDYf//QgN6uHXTtGoYffBCOPjrE/vrr4Vd4NosWhc9xyCHwzjvZlznoIOjRA4YMCd/B+us3bVyH8Gt/1aqwv3nzQuybbBIaqyGse+ihsO662feRWlekweSrJqp5Y3A5r8QbkCtp1Sr3+fNrt/9ly9zPPbeyMSxd6j53bnnrrlzpftVV7g88EBrMwX2PPdwffbTpcosXuz/4oPu4ce7nnx+Wu+oq9+XLw2vcOPcVK8KyX3zhPn16aNzO55NP3C+9NHwn7qHxP1vD88KF5X02kTqHGpClVUtV662+euW3PWcOvP8+fPvb6WlTpoRLWEUajBqQpXUzSyYRQKgq2nPPUJ2Xkrq4QKQNUTIQAbjkkvTw/Pm1i0OkRpQMRFJSd0mrZCBtkJKBSMrAgeF9+vTaxiFSA0oGIil9+oT3444Ll+OKtCFKBiIp7dunh594onZxiNSAkoFINj/7Wa0jEKkqJQMREVEyEBERJQOR3MaNq3UEIlWjZCASN2hQenjy5NrFIVJlSgYicTfdlB7WozKlDVEyEMll+fJaRyBSNUoGIrl89BFstpmqi6RNUDIQyeXaa2HSpPCwIZEGp2QgkkuqRLBqVW3jEKkCJQORQpQMpA1QMhAppBU+DVCkVIkmAzPrZWYvmNlYMxtjZqdkWcbM7FozG29mo81spyRjEimZSgbSBiRdMlgBnO7uWwO7AieZ2TYZy3wf2DJ6DQJuSDgmkdJcdFF4BrNIA0s0Gbj7NHd/OxpeCIwFMp80fjBwpwf/AdYxs42SjEukZLNn1zoCkURVrc3AzHoD3wTezJi1MRC/kHsKzRMGZjbIzEaY2YiZM2cmFaaISJtUlWRgZmsB/wROdffMR0hZllWatdi5+83u3s/d+/Xo0SOJMEVyUyOyNLjEk4GZdSQkgnvc/aEsi0wBesXGNwGmJh2XSEmUDKTBJX01kQFDgLHufnWOxYYBR0VXFe0KzHf3aUnGJZLXI480n6ZkIA2uQ8Lb3w34BfCemY2Kpp0DbArg7jcCTwL7A+OBxcCxCcckkt8uuzSfpmQgDS7RZODur5K9TSC+jAMnJRmHSElWX73WEYhUne5AFsnUrVvzaSoZSINTMhAphpKBNDglAxERUTIQKYpKBtLglAxERETJQCSrLl2ajqtkIA1OyUAkm44dm44rGUiDUzIQySYzGeiZBtLglAxEsumQcT+mSgbS4JQMRLJp377puEoG0uCUDESyUTKQNkbJQCSbzGRw0UWwcmVtYhGpAiUDkWwyk8HQofDgg7WJRaQKlAxEsslMBgDLl1c/DpEqUTIQySZbMminfxdpXPrrFsnmxBObT8uWIEQahJKBSDa/+lXzaSoZSAPTX7dIsVQykAamZCBSLJUMpIHpr1ukWEoG0sD01y1SLFUTSQNTMhAplpKBNDAlA5FitWsHjzwC99xT60hEKq5D4UVEBAglg0MPDcNHHlnbWEQqTCUDkWJlPvBGpIEoGYgUS91YSwNTMhDJZfPNm47raWfSwJQMRHIxazquZCANTMlApFhKBtLAlAxEiqVkIA1MyUCkWGpAlgamZCBSrJNPrnUEIolRMhAp1kcf1ToCkcQkmgzM7DYzm2Fm7+eYv5eZzTezUdHrvCTjESmJ2gikDUm6O4o7gOuBO/Ms84q7/yDhOEREJI9ESwbu/jIwJ8l9iCTmb3+rdQQiVVMPbQYDzOxdM3vKzLbNtZCZDTKzEWY2YubMmdWMT9qq/fardQQiVVPrZPA2sJm77wBcBzySa0F3v9nd+7l7vx49elQtQBGRtqCmycDdF7j7omj4SaCjmXWvZUwiTdyZr7lLpHHUNBmY2YZmoQMYM9slimd2LWMSaeIXv6h1BCJVUfTVRGa2AXAZ0NPdv29m2wAD3H1InnXuA/YCupvZFOB8oCOAu98IHAb80sxWAF8Bh7vrej4RkWor5dLSO4DbgcHR+EfAUCBnMnD3I/Jt0N2vJ1x6KiIiNVRKNVF3d78fWAXg7iuAlYlEJVLvli6tdQQiFVVKMvjSzNYDHMDMdgXmJxKVSL075BC49dZaRyFSMVZsFb2Z7US4/HM74H2gB3CYu49OLrzs+vXr5yNGjKj2bqWtynzITZyauKQVMbOR7t4v27yiSwbu/jbwbeBbwAnAtrVIBCJ15eKLax2BSEWUUjI4Ktt0d6/6hdgqGUhV5SsZgEoH0mrkKxmUcjXRzrHh1YG9CXcQ664cEZFWruhk4O5NnuxhZmsDd1U8IhERqbqW3IG8GNiyUoGIiEjtlHIH8mNEl5USksg2wP1JBCUiItVVSpvBn2PDK4CJ7j6lwvGI1J9u3WDu3FpHIZKoUtoMXkoyEJG6NWcO/OQn8MADtY5EJDEFk4GZLSRdPdRkFuDu3rXiUYnUm3a1fvSHSLIKJgN371KNQETqmpKBNLhS2gwAMLP1CfcZAODukyoakUg9UjKQBlf0X7iZHWRmHwOfAS8BE4CnEopLpL60b1/rCEQSVcrPnYuBXYGP3L0P4Q7k1xKJSqTeqGQgDa6Uv/Dl7j4baGdm7dz9BWDHhOISqS9KBtLgSmkzmGdmawEvA/eY2QzC/QYijU/JQBpcKX/hBxO6oPgt8C/gE+DAJIISqTtqM5AGV0rJYBDwQHTX8d8TikekPqlkIA2ulL/wrsDTZvaKmZ1kZhskFZRI3cmXDHbaCebrCbDSupXypLML3X1b4CSgJ/CSmT2XWGQi9WTo0Nzz3nkHntJV1tK6lVP2nQF8AcwG1q9sOCJ1atasWkcgkqhSbjr7pZm9CPwb6A4c7+7bJxWYSKtS6NGYInWulAbkzYBT3X1Utplm1s3d1c+viEgrVEoX1mcVWOTfwE4tC0dERGqhktfLqZwsItJKVTIZZHvmgYiItAK6k0ZERFRNJFKUhx6qdQQiiSrl0tLNzaxTNLyXmf3GzNaJLbJ3xaMTqRfbblvrCEQSVUrJ4J/ASjPbAhgC9AHuTc109zkVjk2kfhTqm8jVZCatWynJYJW7rwAOBf7i7r8FNkomLJE6o47qpMGV9HAbMzsCOBp4PJrWMd8KZnabmc0ws/dzzDczu9bMxpvZaDPTfQpSn3SHsTS4UpLBscAA4FJ3/8zM+gB3F1jnDmBgnvnfB7aMXoOAG0qIR6R6VDKQBlfKHcgfAL+B0PUE0MXdLy+wzstm1jvPIgcDd7q7A/8xs3XMbCN3n1ZsXCJVoWQgDa6Uq4leNLOuZrYu8C5wu5ld3cL9bwxMjo1PiaaJ1JdC1URz5sB226k6SVqtUn7urO3uC4AfAre7e19gnxbuP9t/TtbLMsxskJmNMLMRM2fObOFuRUpUqGRw0kkwZkx1YhFJQCnJoIOZbQT8hHQDcktNAXrFxjcBpmZb0N1vdvd+7t6vR48eFdq9SJFKrSaaOROefjqZWBqJe3g4kNRcKX/hFwFPA5+4+3Az+xrwcQv3Pww4KrqqaFdgvtoLpC6VWv3zve/BwIGwbFky8TSKe+8Njw198MFaR9LmldKA/ADwQGz8U+BH+dYxs/uAvYDuZjYFOJ/oclR3vxF4EtgfGA8sJlyxJFJ/Si0ZfPBBeNfNaPmljtOHH9Y2Dik+GZjZJsB1wG6Eev1XgVPcfUquddz9iHzbjK4iOqnYGERqptyrifKVKMaMgaVLwy9jkRor5S/8dkK1Tk/CFT+PRdNEGl+5VwktXQqvvJJ93nbbQd++5cckUkGlJIMe7n67u6+IXncAasmVtqGUksHChbB8eRg+4wzYc08YPTqZuEQqpJRkMMvMfm5m7aPXz4HZSQUmUldKSQaHHZYeTiWB2fpXkfpWSjI4jnBZ6RfANOAw1OArbUUp1URvvpkeXrWq9PVFaqDoZODuk9z9IHfv4e7ru/shhBvQRBpfKSWD+fPTw7qaSFqJlna4clpFohCpd9mSwVprFV5v+PDKxyKSgJYmA5V9pW2IV/OYwVVXwckn1y4ekQpraTJQGVjahnh1T6dOcNppMDVrzynZvf++qoykrhW86czMFpL9pG/AGhWPSKQedeoE7dvDypXpUsK4ccWvf/LJ0K0bHHlkMvGJtFDBkoG7d3H3rlleXdy96DuYRVq1du3g88/DcCoZlHqFkO41kDqmJ3aIFCvz5F9qMrjiCl1iKnVLyUCkWOWWCERaASUDkWJlJgMlBWkgSgYixWppNZFIHVMyEClWNUoGy5eH7V5zDcyale7OQiRhSgYixapGMvjyy/B+2mnQowece27l91FPdO9F3VAyEClVNauHHn64evuSNk3JQKRYqV+xqWRQ7tPPitlHrXz1Vfq5zaefDo89luz+1O5SN5QMREpVzauJqp0cOneGb34zDF99NRx0UHX3LzWjZCBSrMwTc4cGvQE/9ZD6Whs3Lv3EOEmckoFIsTKriZJIBrWuJqq2XJ/3889h663h1FOrG08b1qA/bUQSlEoGHTtWdrvvvQe33VbZbbZWc+aE95deqm0cbYiSgUixOnUK77vuGt4rnQwGDEhfWtpW5Gp3UcNy1SkZiBSrSxcYMQK+8Y0wXulqoiVLKrs9kRIoGYiUom/f9HCjNiBD/bRd1EscbYAakEXK1ZL7DJ55Jj1cjyWClSurs59cJ3tVE1WdkoFIucpNBitWwGefpcdfeQWefLIyMVVKPBn83/+Vt43Ro8Nnk1ZByUCkXGtET31NNSgX64wzmv7yPfJIOOCA7L/GU7+cP/ywup3WxWM56aT01T2l2GEH2HPP/MsUKgHkqyZatSokVqkIJQORcl1+eTixl3r541//mu7yAWDmzPzLjxoFW20FV15ZeozlykxM1e49NZUkxo7NXY22++6Vv6KrDVMyECnXOuuEE/Rqq5W+7tlnF7/sxInh/bXXmk4v59d6sSrdZjB9OsyYUd66N9+cffobb5QfT6VssAHcdFOto6gIJQORWli0qLjlli2D9u3DcPwE/dBDsN56zRNEpVQ6GWy4YThxZirmaqFqNWaXyj0kuBNPLG/9l1+G55+vbEwtoGQgUgm5fr221IQJ8OabYTheVfPCC+F9yBDYbLPKlxIyT8C5TtoffBAexJNPOT2ftoariVp62eu3vw17712ZWCpAyUCkEpKsU7/kkvAeP0GnTkS33w6TJsG//51eZujQ9PzFi5u2TxSr2F/ju+wSHsST78SY75kMreGkn0uDPYVOyUCkEqpxc1T85JN5IkqN/+UvcPjhcNddYXzNNaF//9L3VWzJINV9Rr7Pf/vtpe+/NWiwG+ISTwZmNtDMPjSz8WZ2Vpb5x5jZTDMbFb3+N+mYRCquGr8Sx47Nvb/UyXvatPA+fXp63qhRhbe9YgWcfHLz7RWr0ifGeIlhyRL42c9g8uTK7qOlGiwZJHo/vZm1B/4GfA+YAgw3s2Huntlh+lB3/3WSsYgkqhonhqlTw37McpcMUifRUuN55hm4/vr0+D77NJ1faHvlfv5i1nvssXDl0PLl8MAD5e2nFCtXhhsKU8fy1lthjz3SfVKlZB7zVi7pksEuwHh3/9TdlwH/AA5OeJ8i1Vet+uPUw14yT6ItPTFNndp0fPz4puNJJYNcsn2OSZPgN79J5uqiiRPDPu+8M/Q5df756XnHH59++ltcg5UMkk4GGwPxst2UaFqmH5nZaDN70Mx6ZduQmQ0ysxFmNmJmoZt0RKqtWskgVf2TeSK65JLC3V9PnRpOeHfc0Xze8cfnX7eayWDlSrjxxubT33oLrrsOXn+9cvtKSVWlpbreyNz/V181X0clg5JkO0qZfzWPAb3dfXvgOeDv2Tbk7je7ez9379ejR48KhynSQtVKBptumn1/H38M556bv5roo4/CezkNugsX5p9fyWRw222hITyJbeeS2keqv6li9qmSQUmmAPFf+psATcqj7j7b3ZdGo7cAfRFpbX7+8+ruL5UM4t1oL1qUPxmkli2nmiVVX/7EEzBmTPP55Z4Ys/2qnjcv/7aT/CWeefwK9Y2UdDxVlHQyGA5saWZ9zGw14HBgWHwBM9soNnoQMBaR1qZHD7jwwurtL3UiSt2dnFJMMmhJ524/+AFst13z6eUkg003hcsuK7xcNZNBoX0XO68VSjQZuPsK4NfA04ST/P3uPsbMLjKzg6LFfmNmY8zsXeA3wDFJxiSSmMwTc5JSJ6J4ySB+kszWzUElkkGheEpR7qWihZLBzJlw3HHZ6/kffxxefbXwtlOfJ1/1X2qZJJLTwoWFq+YqLPH7DNz9SXf/urtv7u6XRtPOc/dh0fDZ7r6tu+/g7t9x93FJxySSiFNOgRNOSH4/660XuoGApr12jhyZPjE99xw89VR63jPPpJNB6oqkUsU7muvWDc47Lz1eyTaTzJNrqSWDs84K7SL33dd83oEHhstEM2We2ItJBrmqid5+G8aVcBrLtmzXruH1ySfFb6eFdAeySKWstVb2q2Aqbc6ccMKBpiWDt99uemJK3YAGsN9+6QfqjB5d3n5ffDE9PG8eXHxxejzJKpPM3k6zJYP//Cd0/Bafn+9EPmcO7LUXTJmSfX4xbQa55vXtC1tvnX3emDHQsyf8/vfpaVtvnbsUsMUWufdfYUoGIq1Z5tPW4ifKzGqr+fNbtq98XXW3NBnceWfueakuvPMZMCB0/AbpY5Kvofyuu8JzKK64oun0XPdvZFNOA/KFF4YknflsikL9R116Kfzyl8XvpwxKBiKNJH5iykwULb1Zq0OeDgviJ1H30ntxTV32mrmtbAqdfFOfO9+JPHW3dWYJIHPfuWIZMaKyd0OvWhWqGf/61+zzzz038VJnot1RiEjCunVrWo0Sr27ILBm89156eNq00PZQyoN58j1VLH7SfOWV8ttOli8PVT75FEoGqc+dLxlk3mGdkpkUcm1j553zx5BLruSybBlce20YPuWU8rbdQioZiLRmmSeX+C/LzGccxOf17BmuuCnF0qW55/3xjyHBzJ1b/IN7Mr32Wni850MP5V+u2JJBMSUh93Bnc+oO5EIlhXLiydxfNgcdlH06FH5eRIUoGYhU2q23poevuio9vPnmld9XvHolU7wX0mz++c/wXmz1Ua5nEUOoe+/ZE9Zdt7htZbP77vDpp4WXq0Q1UVz//unG8GJLBuXKlQxSFwRkc9pplY0hByUDkUpbY43wfsQR8OMfp6fXazcqv/tdccvlKxlUU75kcPLJ2auJFi8OJYBMuU7O2UoGe+wR+n/KvIejXo5LCykZiFRap07hfcmS/PXstbZkCXz4Idx/f3HLl/PEtCTkSwbXX5+9ZHDUUdkf8pP5yz/f1USvvho69JR7TUQAAA8mSURBVMv2qMpsx2bnncPDhfJtv44oGYhU2uqrh/clS8KD4FPqsQ+brbaCzz8vbtmzz042lkpJ3WwWr/4aPry4dVOlh1zVRNn6ZYLs7S8jRoQSSamy9SpbBUoGIpW2226w/voweHAYT/WFX8e/CotSia7jFywIVS3lNjJD4aSaejZD/ESeeZltSinVRPmWT7W/ZPPaa4XXjzv22MLLJEDJQKTS1lknPHdgt93CeL7O4zbO9niPBrb22tCrF/w9a0/1xSm2hHXOOYXXuemm/NvILBnEL8+Ny9fFxz33pIdb+oMgwSuLlAxEqiXbCalbt+rHkbRCJ6y5c1u2/eHDm3a1kYRSLi2F/Fdk5SqVlOOiiyq3rQxKBiJJO/zw8L7JJrWNo1qee67wMi35hTxoEPTuXdo6pbbXfPUVPPtsaZeW/u532fcTv/mvpSWDBKsalQxEknbGGaGevK0kg2K09Pr9Uq5sOvro4u5fyLTvvqWdfP/85+zT4wmipSfzBC9jVTIQSZoZdO4MF1zQsu187WsVCacuPP545bZVqGE7Xyd4hVTiprN4NVFLk0G+G/9aSMlApFrWXrtp//QHHlja+qn7FxrBs89Wblvrr597XkuuWoLcfRiVYvLk0NFcpap4Mrv0rhB1VCdSTfFf96XekNZIyaBaunRp2frf/W7LY3jwwfD+859XJiGMH58/AZZJJQORWsnXJXQ2Sgat25w5lanmSagRWSUDkVoxS/djVIxSk4fUl9R9Jy11442V21aMSgYi1ZZqK/jGN9JVCMWox+4spPoSuqJIPzVEqm3YsNCA+p3vlPZrX8lAoPlDiypEJQORWvje90pLBPvuq2QgQULVhUoGIq1BpR+yIq2XSgYiDe7ll3PP23bb8ksGLb3ZTeqLSgYiDW6PPXLPu+KK7Mng618vvN1f/KL8mKT+KBmItAGzZjWftvPOsNpq2ZPBNdekr0hKPVQnUzHVCkcdVXyM+Zx5ZmW2I7klVE2kq4lE6sl666W7Q0790++3X/PlTjgBBg6E/fcPD4yBcFfqpEnNly3m5LHeeuXFm6mS3TVLdioZiLQR7dqF1/Dh8Mgj6Tr/VJVQly7hxqNDDgnjXbvCX/4CL77YfFv/+ldxyeCSSyoRef6G7gsvrMw+2rqEnoFh3gofxdevXz8fMWJErcMQqQyz8OD0Qp2qLVkCzz8fSgP5thXnHjo222CD/Nt2r8ylq6efDlddlX3eqlUqOVTCkiVld01iZiPdvV+2efpmRGrt5Zdh3LjCy62+ev5EkEspXV4AzJsXkkMxjdOZVqzIPa+cZFNqZ36VUs5xrrQjjsg+PaE+qpQMRGptjz2SffDNmmuWtvzaa4f3VNvFWmvB1VdXpj+cf/yjtOUnTYIxY4pLCkOHlhdTNt/6VmnL7713+fvaYovs0++7r/xtlkHJQKTR5auaueMOOPLI9Phpp6WHO3cO76NHw29/Cy+8AG+9FaZtvXV6ud/+Nj287bbhfZttsu+ve/eiwwZgww3DtnL9So6bOjVUUxVy1lnwhz+Eq7RyKbVE8sQTpS0f99ln5a9bQUoGIo0os0rmppvgssvCpajxK4eOOgruvjsMuzet73/0Ubj44vTzhjt2hH79wmM8H300vdzOO6eTyN57w9tvw+DB2ePKlpi23LLw5xkyBC69NP8y3/pW/gfTp/TsGR4sn0pc2ZRSpbXRRqHqZtiw4teJyxdzPOkmzd0TfQEDgQ+B8cBZWeZ3AoZG898EehfaZt++fV1Esrj0UvdddnGfNs39o49yLxdO/S3b14IF7pdc4r5ihfvy5e5jxqTnTZmS3kd8X4891nz6xInua60Vhl9+ufk6Kdde23zd+GvlSve//rXptGOPbb7c/feH7b3zTu5tXXll/n2lXpdf7v7ll+kYTznF/eij3T/7rPC6gwa59+/vPn26+/bbu7/3XtP5nTq5f/GFe/v2YXyNNdz79GnRVwaM8Fzn6lwzKvEC2gOfAF8DVgPeBbbJWOZXwI3R8OHA0ELbVTIQaaGOHVueDIpx881NT+xz5zY/KU6Y4H788WF4yRL3HXcMw2ee2XRbb7zRdL1XXkkPp5ZdudL9X/8KyfCEE0KCAvfDDksvu2pVeptXX+2+1Vbuf/qT+8MPuw8eHJZ59tnw/oMfhOUOOCD7Cf2ZZ3J/9rPOyp0IfvjD7OtcdVV6mZdeCtN+9KN0EovHXoZaJoMBwNOx8bOBszOWeRoYEA13AGYRXfKa66VkINJCY8e6DxmS/H5WrHD/5S/d33wzPW3+fPfFi9MnvfffDyWLmTPD/JEj3ffay/2rr5pvL7XOqFFhfNEi91tuyX+SXLEive6xx+aPd9WqkJDc3e+7LyQv91B66d/fvWdP91dfDfGOGJF/WytXhiT0+OMhQcWTwXHH5V7vyy9DMkp56y33dddNH58WyJcMEr3PwMwOAwa6+/9G478A+rv7r2PLvB8tMyUa/yRaZlbGtgYBgwA23XTTvhMnTkwsbhGpgilTQlvGRRcVX0e/eHE4nZZ6hVQ9mDIlNIj/+c/wq1+FmwWrrJb3GWT7hjOzTzHL4O43u3s/d+/Xo0ePigQnIjW0ySahgbqUxtrOnVtnIoDweTt0CFcz1SARFJJ0MpgC9IqNbwJMzbWMmXUA1gbmJByXiIjEJJ0MhgNbmlkfM1uN0ECcef3VMODoaPgw4HlPsu5KRESaSbTXUndfYWa/JjQStwduc/cxZnYRoSFjGDAEuMvMxhNKBIcnGZOIiDSXeBfW7v4k8GTGtPNiw0uAHycdh4iI5KY7kEVERMlARESUDEREBCUDERGhlT7pzMxmAuXegtyd0OVFPav3GOs9PlCMlVDv8UH9x1hv8W3m7lnv2m2VyaAlzGxErtux60W9x1jv8YFirIR6jw/qP8Z6jy9O1UQiIqJkICIibTMZ3FzrAIpQ7zHWe3ygGCuh3uOD+o+x3uP7rzbXZiAiIs21xZKBiIhkUDIQEZG2lQzMbKCZfWhm483srBrF0MvMXjCzsWY2xsxOiaZfYGafm9mo6LV/bJ2zo5g/NLP9qhTnBDN7L4plRDRtXTN71sw+jt67RdPNzK6NYhxtZjslHNs3YsdplJktMLNTa30Mzew2M5sRPb0vNa3kY2ZmR0fLf2xmR2fbV4VjvNLMxkVxPGxm60TTe5vZV7HjeWNsnb7R38f46HOU8ISakuMr+XtN8n89R4xDY/FNMLNR0fSqH8Oy5XoeZqO9CF1ofwJ8DVgNeBfYpgZxbATsFA13AT4CtgEuAM7Isvw2UaydgD7RZ2hfhTgnAN0zpl0BnBUNnwX8KRreH3iK8NS6XYE3q/y9fgFsVutjCOwJ7AS8X+4xA9YFPo3eu0XD3RKOcV+gQzT8p1iMvePLZWznLcIzzi36HN9PML6Svtek/9ezxZgx/yrgvFodw3JfbalksAsw3t0/dfdlwD+Ag6sdhLtPc/e3o+GFwFhg4zyrHAz8w92XuvtnwHjCZ6mFg4G/R8N/Bw6JTb/Tg/8A65jZRlWKaW/gE3fPd0d6VY6hu79M86f0lXrM9gOedfc57j4XeBYYmGSM7v6Mu6+IRv9DeCJhTlGcXd39DQ9ntTtjn6vi8eWR63tN9H89X4zRr/ufAPfl20aSx7BcbSkZbAxMjo1PIf9JOHFm1hv4JvBmNOnXUVH9tlR1ArWL24FnzGykmQ2Kpm3g7tMgJDVg/RrHCOFhSPF/vHo6hlD6Mav13+lxhF+pKX3M7B0ze8nM9oimbRzFlVKNGEv5Xmt5DPcAprv7x7Fp9XIM82pLySBbfVzNrqs1s7WAfwKnuvsC4AZgc2BHYBqhqAm1i3s3d98J+D5wkpntmWfZmsRo4VGqBwEPRJPq7RjmkyummsVqZoOBFcA90aRpwKbu/k3gNOBeM+tagxhL/V5r+X0fQdMfJ/VyDAtqS8lgCtArNr4JMLUWgZhZR0IiuMfdHwJw9+nuvtLdVwG3kK7GqEnc7j41ep8BPBzFMz1V/RO9z6hljIRE9ba7T49iratjGCn1mNUk1qih+gfAkVG1BVH1y+xoeCShHv7rUYzxqqREYyzje63VMewA/BAYmppWL8ewGG0pGQwHtjSzPtEvysOBYdUOIqpTHAKMdferY9PjdeyHAqkrFYYBh5tZJzPrA2xJaHhKMsY1zaxLapjQwPh+FEvq6pajgUdjMR4VXSGzKzA/VTWSsCa/wurpGMaUesyeBvY1s25Rdci+0bTEmNlA4EzgIHdfHJvew8zaR8NfIxy3T6M4F5rZrtHf81Gxz5VEfKV+r7X6X98HGOfu/63+qZdjWJRatl5X+0W4guMjQnYeXKMYdicUB0cDo6LX/sBdwHvR9GHARrF1Bkcxf0gVrjggXIXxbvQakzpWwHrAv4GPo/d1o+kG/C2K8T2gXxVi7AzMBtaOTavpMSQkpmnAcsIvv/9XzjEj1NuPj17HViHG8YQ69tTf443Rsj+Kvv93gbeBA2Pb6Uc4KX8CXE/Um0FC8ZX8vSb5v54txmj6HcCJGctW/RiW+1J3FCIi0qaqiUREJAclAxERUTIQERElAxERQclARERQMhDJy8wGW+hddnTU62R/Cz2kdq51bCKVpEtLRXIwswHA1cBe7r7UzLoTesF8nXBfwKyaBihSQSoZiOS2ETDL3ZcCRCf/w4CewAtm9gKAme1rZm+Y2dtm9kDU71TqmRB/MrO3otcW0fQfm9n7Zvaumb1cm48m0pRKBiI5RCf1Vwl3Oz8HDHX3l8xsAlHJICotPES4+/VLMzsT6OTuF0XL3eLul5rZUcBP3P0HZvYeMNDdPzezddx9Xk0+oEiMSgYiObj7IqAvMAiYCQw1s2MyFtuV8JCV1yw83epowoN2Uu6LvQ+Ihl8D7jCz4wkPYhGpuQ61DkCknrn7SuBF4MXoF33mYyiN8DCaI3JtInPY3U80s/7AAcAoM9vRo54tRWpFJQORHCw8a3nL2KQdgYnAQsIjSyE8GWy3WHtAZzP7emydn8be34iW2dzd33T384BZNO1uWaQmVDIQyW0t4DoLD4hfQejdcxCh6+ynzGyau38nqjq6z8w6ReudS+gxE6CTmb1J+OGVKj1cGSUZI/Rk+m5VPo1IHmpAFklIvKG51rGIFKJqIhERUclARERUMhAREZQMREQEJQMREUHJQEREUDIQERHg/wOEq/MRFcBnQQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "steps = step_loss[\"step\"]\n",
    "loss_value = step_loss[\"loss_value\"]\n",
    "steps = list(map(int, steps))\n",
    "loss_value = list(map(float, loss_value))\n",
    "plt.plot(steps, loss_value, color=\"red\")\n",
    "plt.xlabel(\"Steps\")\n",
    "plt.ylabel(\"Loss_value\")\n",
    "plt.title(\"Loss function value change chart\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面可以看出来大致分为三个阶段：\n",
    "\n",
    "阶段一：开始训练loss值在2.2上下浮动，训练收益感觉并不明显。\n",
    "\n",
    "阶段二：训练到某一时刻，loss值减少迅速，训练收益大幅增加。\n",
    "\n",
    "阶段三：loss值收敛到一定小的值后，loss值开始振荡在一个小的区间上无法趋0，再继续增加训练并无明显收益，至此训练结束。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  数据测试验证模型精度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "搭建测试网络的过程主要为：\n",
    "\n",
    "1. 载入模型`.cptk`文件中的参数`param`；\n",
    "2. 将参数`param`载入到神经网络LeNet5中；\n",
    "3. 载入测试数据集；\n",
    "4. 调用函数`model.eval`传入参数测试数据集`ds_eval`，就生成模型`checkpoint_lenet-{epoch}_1875.ckpt`的精度值。\n",
    "\n",
    "> `dataset_sink_mode`表示数据集下沉模式，不支持CPU，所以这里设置成`False`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:58.668334Z",
     "start_time": "2020-09-04T06:46:57.781766Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "============== Starting Testing ==============\n",
      "============== Accuracy:{'Accuracy': 0.9653445512820513} ==============\n"
     ]
    }
   ],
   "source": [
    "# testing relate modules \n",
    "def test_net(network, model, mnist_path):\n",
    "    \"\"\"Define the evaluation method.\"\"\"\n",
    "    print(\"============== Starting Testing ==============\")\n",
    "    # load the saved model for evaluation\n",
    "    param_dict = load_checkpoint(\"checkpoint_lenet-1_1875.ckpt\")\n",
    "    # load parameter to the network\n",
    "    load_param_into_net(network, param_dict)\n",
    "    # load testing dataset\n",
    "    ds_eval = create_dataset(os.path.join(mnist_path, \"test\"))\n",
    "    acc = model.eval(ds_eval, dataset_sink_mode=False)\n",
    "    print(\"============== Accuracy:{} ==============\".format(acc))\n",
    "\n",
    "test_net(network, model, mnist_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过1875步训练后生成的模型精度超过95%，模型优良。\n",
    "我们可以看一下模型随着训练步数变化，精度随之变化的情况。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`eval_show`将绘制每25个`step`与模型精度值的折线图，其中`steps_eval`存储着模型的step数和对应模型精度值信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:58.821007Z",
     "start_time": "2020-09-04T06:46:58.671515Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3debwcVZn/8c+XhIQQlgBJMGQhiQYQQbYYQREVAQERGARCGNyV0RGVUWdkRn/I4KijuKAjM4jKDLjkhlVQ2QSJqHCbJEAgEJYQwIQ1rIEEEpI8vz9ONenc9L23bnK7q5fv+/WqV1fX1k9X31tPnXOqTikiMDOz9rVJ0QGYmVmxnAjMzNqcE4GZWZtzIjAza3NOBGZmbc6JwMyszTkRWJ9IGi8pJA3MsexHJP2lHnG1C0l/L+m6jVj/akkf7s+Yevm83H8vVhwnghYm6WFJKyUN7zL9juyfc3wxkdmGiohfRcQheZaVdIakX3ZZ/7CIuKA20dVH9rf7hqLjaCVOBK3vIWBa+Y2k3YEhxYXTGJrxDLUZY+5P7f79a8mJoPX9AvhQxfsPAxdWLiBpa0kXSloi6RFJX5W0STZvgKTvSnpa0kLgfVXW/bmkxyU9Kuk/JA3IE5ikiyU9IekFSTdJelPFvCGSvpfF84Kkv0gaks3bX9LNkp6XtEjSR7LpMyV9omIb61RNZWeSn5H0APBANu2H2TaWSpoj6R0Vyw+Q9G+SHpT0YjZ/rKRzJH2vy3f5raRTq3zHcyV9t8u0KyR9IRs/rWL790j6uy7x/1XSDyQ9C5xR5TtVjV/SocC/AVMlvSRpbtd9JGmT7Ld+RNJT2d/A1tm8cpXOhyX9Lfv9v9LDb9nt75X5+2rbkTRF0i3Zb/m4pB9LGtTdbybppmzW3Ox7Te0uJuuDiPDQogPwMHAQcB/wRmAAsAjYEQhgfLbchcAVwJbAeOB+4OPZvE8B9wJjgW2BG7N1B2bzfwP8BBgKjARuBf4hm/cR4C89xPex7DMHA2cDd1TMOweYCYzO4n5bttw44EVSKWdTYDtgz2ydmcAnKraxzudncf8h+x5DsmknZdsYCHwReALYLJv3z8BdwM6AgD2yZacAjwGbZMsNB5YD21f5jgdk+1zZ+22Al4EdsvfHATuQTsqmAsuAURXxrwI+m8U3pMp36in+M4BfdonntX2U7f8FwERgC+Ay4BfZvPHZ/vpp9rl7ACuAN3bzW3b3e/W4HWAfYN8s/vHAfODUXn6zAN5Q9P9XKw2FB+Chhj/u2kTwVeBbwKHZP9XA7J9pfPZPuwLYtWK9fwBmZuN/BD5VMe+QbN2BwPbZukMq5k8DbszG1zlo9RLrsGy7W2cHxZeBPaos96/A5d1s47WDXLXPz7Z/YC9xPFf+XFICPaqb5eYDB2fjpwBXdbOcgL8BB2TvPwn8sYfPv6P8mVn8f+syv8d92iX+M+g5EdwA/GPFvJ2BVysOygGMqZh/K3BClc/s6ffKvZ1s3qmVv2+13wwngn4fXDXUHn4BnEg6iFzYZd5wYBDwSMW0R0hndpDOVhd1mVe2I+ms/PGsaP88qXQwsreAsmqX/8yqRZaSklY5nuHAZsCDVVYd2830vCq/C5K+KGl+Vp3xPCkRlRvXe/qsC0hn42Svv6i2UKQjVwdr22lOBH5V8fkfUmq8L++/3So+f714u+ol/t7swPq/eznBlz1RMb6cVHLoqqffq8ftSNpJ0u+yKsKlwDerxN/jPrCN50TQBiLiEVKj8eGk4n+lp0lngTtWTBsHPJqNP046IFbOK1tEKhEMj4hh2bBVRLyJ3p0IHEUqsWxNOnOEdAb9NPAK8Poq6y3qZjqkapXNK96/rsoyr3W3m9Wnfxk4HtgmIoYBL2Qx9PZZvwSOkrQHqdrtN90sBzAdOFbSjsBbgUuzz9+RVGVyCrBd9vnzKj5/nXi7yhF/b10LP8b6v/sq4Mle1uuqp9+rN/9DqnqcFBFbkdo11GUZd5FcY04E7ePjpCL2ssqJEbEauAj4hqQts4PTF0gHOrJ5n5M0RtI2wGkV6z4OXAd8T9JWWePj6yW9M0c8W5KSyDOkg/c3K7a7Bjgf+L6kHbLSw36SBpPOpg+SdLykgZK2k7RntuodwDGSNle6vPDjOWJYBSwBBko6HdiqYv7PgK9LmqTkzZK2y2JcDMwilQQujYiXu/uQiLg9+4yfAddGxPPZrKGkg9wSAEkfJZUI8uot/ieB8coa/quYDvyTpAmStiD9BjMiYlUfYujt98rzHZYCL0naBfh0jnWeJLVrWD9xImgTEfFgRMzuZvZnSWfTC4G/AL8m/WNDOmO9FpgL3Mb6JYoPkaqW7iHVT18CjMoR0oWkqohHs3U7u8z/EqmhdhbwLPBtUuPs30glmy9m0+8gNUAC/ABYSTpQXEBFFUw3rgWuJjWOP0I6q62shvg+KRFeRzpY/Zx1L729ANidbqqFuphOKv38ujwhIu4BvgfcksW8O/DXHNvKG//F2eszkm6rsv75Wew3kUqMr5D+FjZE1d8r53onki4A+CkwI8c6ZwAXZNVpx29QtLaO8pUMZtZHkg4glZzGZ2fFZk3JJQKzDSBpU+DzwM+cBKzZORGY9ZGkNwLPk6rAzi44HLON5qohM7M2V7MSgaTzs9vW53UzX5J+JGmBpDsl7V2rWMzMrHu17MTp/4Afs/4NTGWHAZOy4a2k64nf2ttGhw8fHuPHj++fCM3M2sScOXOejogR1ebVLBFExE3quZvjo4ALszsvOyUNkzQquza9W+PHj2f27O6ugjQzs2okPdLdvCIbi0ez7jXPi1nbrcE6JJ0sabak2UuWLKlLcGZm7aLIRND1NnLo5lbyiDgvIiZHxOQRI6qWbMzMbAMVmQgWs24fNmNIfZ+YmVkdFZkIrgQ+lF09tC/wQm/tA2Zm1v9q1lgsaTrwLmC4pMXA10hdFhMR5wJXkfqMWUDqlvajtYrFzMy6V8urhqb1Mj+Az9Tq883MLB93MWFm1uZqeUOZmVnrevVVeOYZePrpNFSOr1hRm898//vhLW/p9806EZiZvfoqPPvs2gN51wN7tfdLl3a/PVW7Or4f7LCDE4GZWS6vvAJLlqThqae6Hy8f1F94ofttDR0Kw4evHSZNgu22W3da5fvttoPBeR7O1jicCMys8VUe2Hs7uC9ZAi++WH07m24KI0asHSZOXPeAXu2gvtlm9f2uBXAiMLP+sWYNLF8Oy5bBSy+l18rxatPyzl/VzWOUqx3Yy+MjR64/vvXWtau2aWJOBGbtaPXqVB3y0kvdD+WDcN55y5b1LYbBg2GLLVLVS/l16NBUD14er5xfecD3gb1fORGYtYvVq+Gmm2DGDLj00lQ3nsdmm609GFcOw4evP63rwbvrAb08vvnmMNCHn0bhX8Ksla1ZAzffDB0dcMkl8OST6SD8/vfDvvvClluufzDvemD3Abvl+Rc2azURcOut6cz/oovg0UfTWf3hh8PUqfC+96UDvFnGicCsFUTA7bevPfg//HBqSD30UPj2t+HII9PZv1kVTgRmzSoC5s1LB/8ZM2DBglSNc9BB8LWvwdFHw7BhRUdpTcCJwKzZ3Hvv2oP//PmwySbw7nfDv/wLHHNMuvbdrA+cCMyawYMPrj3433lnumTyHe+AU06BD3wAtt++6AitiTkRmDWiVatg1iz4wx/gyithzpw0fb/94Oyz4dhjYXTVR3yb9ZkTgVkjiID77oPrr08H/5kzU6dmEkyeDGedBccfD+PGFR2ptSAnArOiPPkk3HBDOvBffz0sXpymT5gAJ5yQGn0PPNB1/lZzTgRm9bJsWbqzt3zWf9ddafo228B73gMHH5wO/hMnFhuntR0nArNaWbUq1e2XD/w335z6vR88GPbfH771rXTw33NPGDCg6GitjTkRmPWXiHQtf7mq549/XNvP/d57wz/9Uzrj339/GDKk2FjNKjgRmPWHhQvT2f3Chen9jjvCccelaQcemDpoM2tQTgRm/eGqq1ISOPvs1JfP61/v7pGtaTgRmPWHzs7Uj/7nP190JGZ9tknRAZi1hFIJ3vrWoqMw2yBOBGYb65lnUiPxvvsWHYnZBnEiMNtYpVJ6dYnAmpQTgdnGKpVSD6D77FN0JGYbxInAbGN1dsLuu6dHO5o1IScCs42xZk16LKSrhayJORGYbYwHHoDnn3cisKbmRGC2MTo706uvGLIm5kRgtjFKJdhqK9hll6IjMdtgTgRmG6NUgilT0lVDZk3Kf71mG2r5cpg71+0D1vScCMw21G23werVTgTW9JwIzDZUuaHYicCanBOB2YYqldLzhUeOLDoSs41S00Qg6VBJ90laIOm0KvPHSbpR0u2S7pR0eC3jMetX7nHUWkTNEoGkAcA5wGHArsA0Sbt2WeyrwEURsRdwAvDftYrHrF899hgsWuT7B6wl1LJEMAVYEBELI2Il0AEc1WWZALbKxrcGHqthPGb9xz2OWgupZSIYDSyqeL84m1bpDOAkSYuBq4DPVtuQpJMlzZY0e8mSJbWI1axvOjth0CDYa6+iIzHbaLVMBNUe2Bpd3k8D/i8ixgCHA7+QtF5MEXFeREyOiMkjRoyoQahmfVQqwZ57wuDBRUdittFqmQgWA2Mr3o9h/aqfjwMXAUTELcBmwPAaxmS28VatgtmzXS1kLaOWiWAWMEnSBEmDSI3BV3ZZ5m/AewAkvZGUCFz3Y43t7rth2TI3FFvLqFkiiIhVwCnAtcB80tVBd0s6U9KR2WJfBD4paS4wHfhIRHStPjJrLG4othYzsJYbj4irSI3AldNOrxi/B3h7LWMw63elEgwfDhMnFh2JWb/wncVmfdXZmUoDqnY9hFnzcSIw64sXXoD5810tZC3FicCsL2bNggg3FFtLcSIw64tyQ/Fb3lJsHGb9yInArC9KpfRYymHDio7ErN84EZjlFZEail0tZC3GicAsr4cfhiVL3FBsLceJwCwv30hmLcqJwCyvzk4YMgR2373oSMz6lROBWV6lEkyeDANrekO+Wd05EZjlsWIF3Habq4WsJTkRmOUxdy6sXOkrhqwlORGY5eGGYmthTgRmeXR2wujRMGZM0ZGY9TsnArM8SiWXBqxlORGY9ebpp+HBB50IrGU5EZj1ptw+4IZia1FOBGa9KZVgwADYZ5+iIzGriV4TgaTZkj4jaZt6BGTWcDo7YbfdYOjQoiMxq4k8JYITgB2AWZI6JL1X8jP6rE2sWQO33upqIWtpvSaCiFgQEV8BdgJ+DZwP/E3Sv0vattYBmhXq/vvT4yndUGwtLFcbgaQ3A98DzgIuBY4FlgJ/rF1oZg2gszO9ukRgLazX3rMkzQGeB34OnBYRK7JZJUlvr2VwZoUrlWDrrWHnnYuOxKxm8nSjeFxELKw2IyKO6ed4zBpLqZSeT7yJL7Cz1pXnr/sTkl57QKukbST9Rw1jMmsMy5fDnXe6WshaXp5EcFhEPF9+ExHPAYfXLiSzBjFnDqxe7YZia3l5EsEASYPLbyQNAQb3sLxZayg3FDsRWIvL00bwS+AGSf8LBPAx4IKaRmXWCEolmDgRRowoOhKzmuo1EUTEdyTdBbwHEPD1iLi25pGZFa1Ugne8o+gozGou18NXI+Jq4Ooax2LWOB59FBYvdkOxtYU8fQ3tK2mWpJckrZS0WtLSegRnVhg/kczaSJ7G4h8D04AHgCHAJ4D/qmVQZoUrlWDQINhzz6IjMau5vFVDCyQNiIjVwP9KurnGcZkVq7MT9toLBvsCOWt9eRLBckmDgDskfQd4HHB/vNa6Vq2C2bPhE58oOhKzushTNfTBbLlTgGXAWOADtQzKrFDz5qW7it0+YG2ixxKBpAHANyLiJOAV4N/rEpVZkfxoSmszPZYIsjaBEVnVkFl7KJVg+HCYMKHoSMzqIk8bwcPAXyVdSaoaAiAivt/bipIOBX4IDAB+FhH/WWWZ44EzSHctz42IE3NFblYrnZ2pWsgP4rM2kScRPJYNmwBb5t1wVq10DnAwsJj0qMsrI+KeimUmAf8KvD0inpM0si/Bm/W7F16Ae++FE30+Yu0jTxcTG9ouMAVYUH6WgaQO4CjgnoplPgmck/VoSkQ8tYGfZdY/Zs2CCDcUW1vJ84SyG0nVNuuIiAN7WXU0sKji/WKg63/XTtln/JVUfXRGRFxTJYaTgZMBxo0b11vIZhuuszNVCU2ZUnQkZnWTp2roSxXjm5EuHV2VY71qFaxdE8pAYBLwLmAM8GdJu1U+/wAgIs4DzgOYPHnyeknJrN+USrDLLunxlGZtIk/V0Jwuk/4q6U85tr2YdM9B2RhSW0PXZToj4lXgIUn3kRLDrBzbN+tfEalEcMQRRUdiVld5Op3btmIYLum9wOtybHsWMEnShOzy0xOAK7ss8xvg3dnnDCdVFVV9PrJZzT30EDz9tO8fsLaTp2poDqlKR6QqoYeAj/e2UkSsknQKcC2p/v/8iLhb0pnA7Ii4Mpt3iKR7gNXAP0fEMxv2Vcw2knsctTaVp2pog++qiYirgKu6TDu9YjyAL2SDWbE6O2HzzWG33YqOxKyu8lQNfUbSsIr320j6x9qGZVaAUgkmT4aBuTrlNWsZeTqd+2TlVTzZNf+frF1IZgVYsQJuv93VQtaW8iSCTaS199pndwy77yFrLXfcAStXuqHY2lKeMvC1wEWSziU1Gn8KWO+mL7Om5oZia2N5EsGXSXf1fpp05dB1wM9qGZRZ3XV2wujRaTBrM3kSwRDgpxFxLrxWNTQYWF7LwMzqqlRytZC1rTxtBDeQkkHZEOD62oRjVoAlS2DhQlcLWdvKkwg2i4iXym+y8c1rF5JZnbl9wNpcnkSwTNLe5TeS9gFerl1IZnVWKsGAAbDPPkVHYlaIPG0EpwIXSyp3GDcKmFq7kMzqrFSC3XeHoUOLjsSsEHm6mJglaRdgZ9JVQ/dmvYWaNb81a1IimDat6EjMCpP3XvqdgV1JzyPYSxIRcWHtwjKrk/vug6VLfcWQtbU8Tyj7GunBMbuSOpA7DPgL4ERgza+zM726odjaWJ7G4mOB9wBPRMRHgT1I9xGYNb9SKT2NbOedi47ErDB5EsHLEbEGWCVpK+ApYGJtwzKrk1IpPZ94kzz/CmatKc9f/+ysG+qfkh5Scxtwa02jMquHZcvgzjtdLWRtL89VQ+VnD5wr6Rpgq4i4s7ZhmdXBnDnpqiE3FFub69MTOCLi4RrFYVZ/5YbiKVOKjcOsYK4YtfZVKsHEiTBiRNGRmBXKicDal3scNQN6qBqStG1PK0bEs/0fjlmdLF4Mjz7qhmIzem4jmEN6IpmqzAt8Cak1s3KPoy4RmHWfCCJiQj0DMaurUgkGDYI99ig6ErPC9dpGoOQkSf8vez9Oki+zsObW2Ql77QWDfZO8WZ7G4v8G9gNOzN6/CJxTs4jMam3VKpg929VCZpk89xG8NSL2lnQ7QEQ8J2lQjeMyq5277oKXX3ZDsVkmT4ng1eyB9QEgaQSwpqZRmdWSH01pto48ieBHwOXASEnfIHVB/c2aRmVWS6VSuolsgq+HMIN8fQ39StIcUlfUAo6OiPk1j8ysVjo7U2lA1a6MNms/eW8oewqYXjnPN5RZU3rsMbj3XvjgB4uOxKxh5L2hbBzwXDY+DPgb4HK1NZ+LL06vxxxTbBxmDaTbNoKImBARE4FrgfdHxPCI2A44ArisXgGa9avp02HPPWGXXYqOxKxh5GksfktEXFV+ExFXA++sXUhmNfLQQ6mh+IQTio7ErKHkuY/gaUlfBX5Jqio6CXimplGZ1cKMGel16tRi4zBrMHlKBNOAEaRLSH8DjMymmTWXjg7Ybz8YP77oSMwaSp7LR58FPp89uH5NRLxU+7DM+tn8+TB3Lvzwh0VHYtZw8nQ6t3vWvcRdwN2S5kjarfahmfWjGTPSfQPHHVd0JGYNJ0/V0E+AL0TEjhGxI/BF4Lw8G5d0qKT7JC2QdFoPyx0rKSRNzhe2WR9EpGqhd70LRo0qOhqzhpMnEQyNiBvLbyJiJjC0t5Wy/onOAQ4DdgWmSdq1ynJbAp8DSjljNuubuXPhvvt8tZBZN/IkgoWS/p+k8dnwVeChHOtNARZExMKIWAl0AEdVWe7rwHeAV3JHbdYXHR0wcKBvIjPrRp5E8DHSVUOXka4cGgF8NMd6o4FFFe8XZ9NeI2kvYGxE/K6nDUk6WdJsSbOXLFmS46PNMuVqoYMPhuHDi47GrCHluWroOVLVTV9196zjNFPaBPgB8JEcMZxH1i4xefLk6GVxs7U6O+GRR+DMM4uOxKxh9dTp3JU9rRgRR/ay7cXA2Ir3Y4DHKt5vCewGzFTqBfJ1wJWSjoyI2b1s2yyfjo70OMqjjy46ErOG1VOJYD9S1c50UkNuX/vsnQVMkjQBeBQ4gbWPuyQiXgBeK6tLmgl8yUnA+s3q1XDRRfC+98FWWxUdjVnD6ikRvA44mHQX8YnA74HpEXF3ng1HxCpJp5A6rRsAnB8Rd0s6E5gdET2WOMw22k03wRNP+Gohs150mwgiYjVwDXCNpMGkhDBT0pkR8V95Np51VndVl2mnd7Psu/IGbZZLRwcMHZpKBGbWrR4bi7ME8D5SEhhPemylu6C2xvfqq3DJJXDUUbD55kVHY9bQemosvoDUmHs18O8RMa9uUZltrOuvh2efdbWQWQ49lQg+CCwDdgI+p7XPdxUQEeHWN2tcHR0wbBgcckjRkZg1vJ7aCPLcbGbWeF55BS6/PHUwN3hw0dGYNTwf7K31XH01vPiiq4XMcnIisNYzfTqMHAnvfnfRkZg1BScCay0vvgi/+12qFhqY50msZuZEYK3lt7+Fl192tZBZHzgRWGvp6IAxY+Btbys6ErOm4URgreO55+Caa2DqVNjEf9pmefm/xVrH5ZenO4pdLWTWJ04E1jo6OuD1r4d99ik6ErOm4kRgreGpp+CGG1JpQH3tMd2svTkRWGu45BJYs8bVQmYbwInAWkNHB7zpTbDbbkVHYtZ0nAis+S1aBH/+M0ybVnQkZk3JicCa30UXpdepU4uNw6xJORFY8+vogMmT4Q1vKDoSs6bkRGDNbcECmD3bjcRmG8GJwJrbjBnp9fjji43DrIk5EVhz6+iA/feHsWOLjsSsaTkRWPOaNy8NrhYy2yhOBNa8ZsxIncsde2zRkZg1NScCa04RqVrowANh++2LjsasqTkRWHO67bZ0xZCrhcw2mhOBNafp02HTTeGYY4qOxKzpORFY81mzJrUPHHoobLNN0dGYNT0nAms+N98Mixe7WsisnzgRWPPp6IAhQ+DII4uOxKwlOBFYc1m1Ci6+GI44ArbYouhozFqCE4E1l5kz09PIXC1k1m+cCKy5dHTAllvCYYcVHYlZy3AisOaxciVceikcfXRqIzCzfuFEYM3juuvg+eddLWTWz5wIrHlMnw7bbgsHH1x0JGYtxYnAmsPy5XDFFamDuU03LToas5ZS00Qg6VBJ90laIOm0KvO/IOkeSXdKukHSjrWMx5rY738Py5a5WsisBmqWCCQNAM4BDgN2BaZJ2rXLYrcDkyPizcAlwHdqFY81uY4OeN3r4IADio7ErOXUskQwBVgQEQsjYiXQARxVuUBE3BgRy7O3ncCYGsZjzWrp0lQiOP54GDCg6GjMWk4tE8FoYFHF+8XZtO58HLi6hvFYs7riClixwtVCZjUysIbbVpVpUXVB6SRgMvDObuafDJwMMG7cuP6Kz5pFRwfsuCPsu2/RkZi1pFqWCBYDlU8UHwM81nUhSQcBXwGOjIgV1TYUEedFxOSImDxixIiaBGsN6pln0v0DU6eCqp1bmNnGqmUimAVMkjRB0iDgBODKygUk7QX8hJQEnqphLNasLrssdTTnaiGzmqlZIoiIVcApwLXAfOCiiLhb0pmSyv0HnwVsAVws6Q5JV3azOWtXHR2w886w555FR2LWsmrZRkBEXAVc1WXa6RXjB9Xy863JPf443HgjnH66q4XMash3FlvjuvhiiEjtA2ZWMzUtEZhtkNWr4ac/hTPOSFVCb3xj0RGZtTSXCKyx3HILTJkCn/407LFHeki9mdWUE4E1hiefhI9+FN72tjTe0QF//CPstFPRkZm1PCcCK9aqVfDDH6YD/q9+BV/+Mtx7r+8bMKsjtxFYcf70JzjlFJg3Dw45BH70o3SpqJnVlUsEVn+PPgonngjvehe8+CJcfjlcc42TgFlBnAisflauhO98Jx3wL7sMvvY1uOee9AxiVwOZFcZVQ1Yf110Hn/0s3H8/HHkk/OAHMHFi0VGZGS4RWK098gh84APw3vfCmjXpuQJXXOEkYNZAnAisNl55Bb7+ddhlF7j6avjGN1Kj8OGHFx2ZmXXhqiHrf7/9LZx6KixcCMcdB9/9Lvg5EmYNyyUC6z8LFsARR6Q2gMGD4frr4aKLnATMGlz7lAgWLUpnqGXR5WFple97mlft/dChMHZserj6wAbcpRHw3HNpHzzxROrLB9ZeqdMfr9dfD2edBYMGpRLA5z4Hm25au+9kZv2mAY9aNdLRAf/yL7X9jAEDYNSolBTGjEmvleNjxqRk0d8PYF+6NB3kqw2LF6fX5cv79zOrOemkdHnoqFG1/ywz6zftkwimToXJk9ed1vXa9cr3Pc3r+r7yQFw+8M6dC7/7Hbz88rrrDRwIO+xQPVGUx7ffHjbJau2WL+/+4F4eli5dP7ZyQtp999RAW97+qFHpTL1cqolYd7y31+7mbb897LorZtZ82icRjBtX/7rqCHj22XUP3OXxxYthzpx0KeUrr6y7XjlZvPRSWr+rkSPTQX3SJDjwwPUTyg47uFrGzHJrn0RQBAm22y4Ne+xRfZmI9ID2rmf6jz4KW2yxbmlh7FgYPRo226y+38PMWpoTQdEkGD48DXvtVXQ0ZtaGfPmomVmbcyIwM2tzTgRmZm3OicDMrM05EZiZtTknAjOzNudEYGbW5pwIzMzanKJrT5oNTtIS4JGi4+hiOPB00UH0QTPF61hrp5nibaZYoTHj3TEiRlSb0XSJoBFJmh0Rk3tfsjE0U7yOtXaaKd5mihWaL15XDZmZtTknAjOzNudE0D/OKzqAPmqmeB1r7TRTvM0UKzRZvG4jMDNrcy4RmN4g8BQAAAc2SURBVJm1OScCM7M250TQC0ljJd0oab6kuyV9Ppt+hqRHJd2RDYdXrPOvkhZIuk/SewuI+WFJd2Vxzc6mbSvpD5IeyF63yaZL0o+yeO+UtHcd49y5Yv/dIWmppFMbad9KOl/SU5LmVUzr876U9OFs+QckfbiOsZ4l6d4snsslDcumj5f0csU+PrdinX2yv58F2fdRtc+rUbx9/u0lHZpNWyDptDrGOqMizocl3ZFNL3zf9llEeOhhAEYBe2fjWwL3A7sCZwBfqrL8rsBcYDAwAXgQGFDnmB8GhneZ9h3gtGz8NODb2fjhwNWAgH2BUkH7eQDwBLBjI+1b4ABgb2Dehu5LYFtgYfa6TTa+TZ1iPQQYmI1/uyLW8ZXLddnOrcB+2fe4Gjisjvu2T799NjwITAQGZcvsWo9Yu8z/HnB6o+zbvg4uEfQiIh6PiNuy8ReB+cDoHlY5CuiIiBUR8RCwAJhS+0h7dRRwQTZ+AXB0xfQLI+kEhkkaVUB87wEejIie7hqv+76NiJuAZ6vE0Zd9+V7gDxHxbEQ8B/wBOLQesUbEdRGxKnvbCYzpaRtZvFtFxC2RjlwXsvb71TzeHnT3208BFkTEwohYCXRky9Yt1uys/nhgek/bqOe+7Ssngj6QNB7YCyhlk07Jitznl6sHSEliUcVqi+k5cdRCANdJmiPp5Gza9hHxOKTkBozMpjdCvAAnsO4/UqPuW+j7vmyUuD9GOgstmyDpdkl/kvSObNpoUnxlRcTal9++EfbtO4AnI+KBimmNum+rciLISdIWwKXAqRGxFPgf4PXAnsDjpKIhpCJfV/W+RvftEbE3cBjwGUkH9LBs4fFKGgQcCVycTWrkfduT7uIrPG5JXwFWAb/KJj0OjIuIvYAvAL+WtBXFx9rX377oeAGmse5JTKPu2245EeQgaVNSEvhVRFwGEBFPRsTqiFgD/JS1VRSLgbEVq48BHqtnvBHxWPb6FHB5FtuT5Sqf7PWpbPHC4yUlrNsi4klo7H2b6eu+LDTurHH6CODvsyoJsiqWZ7LxOaR69p2yWCurj+oa6wb89kXv24HAMcCM8rRG3bc9cSLoRVb/93NgfkR8v2J6ZT363wHlqwmuBE6QNFjSBGASqYGoXvEOlbRleZzUWDgvi6t8tcqHgSsq4v1QdsXLvsAL5WqPOlrnjKpR922Fvu7La4FDJG2TVXUckk2rOUmHAl8GjoyI5RXTR0gakI1PJO3LhVm8L0raN/vb/1DF96tHvH397WcBkyRNyEqWJ2TL1stBwL0R8VqVT6Pu2x4V3Vrd6AOwP6n4didwRzYcDvwCuCubfiUwqmKdr5DOAu6jzlcFkK6emJsNdwNfyaZvB9wAPJC9bptNF3BOFu9dwOQ6x7s58AywdcW0htm3pAT1OPAq6Yzu4xuyL0n18wuy4aN1jHUBqQ69/Ld7brbsB7K/j7nAbcD7K7YzmXQAfhD4MVkPBHWKt8+/ffb/eH827yv1ijWb/n/Ap7osW/i+7evgLibMzNqcq4bMzNqcE4GZWZtzIjAza3NOBGZmbc6JwMyszTkRWFtQ6tV086Lj6EnWa+W83pc0619OBNYuTiXds9CysrtczfrMicBaSnZn9e8lzZU0T9JUSZ8DdgBulHRjttwhkm6RdJuki7O+pMrPcvi2pFuz4Q1VPuOMrEO0mZIWZttf74xe0pcknZGNz5T0A0k3KT3b4i2SLlN6PsF/VGx+oKQLsk7XLimXYpT6sf9T1pHgtRVdXMyU9E1JfwI+X5Odai3PicBazaHAYxGxR0TsBlwTET8i9eny7oh4t6ThwFeBgyJ1zjeb1DlY2dKImEK68/Psbj5nF1L30lOAr2X9UfVmZUQcAJxL6lrgM8BuwEckbZctszNwXkS8GVgK/GO27f8Cjo2IfYDzgW9UbHdYRLwzIr6H2QZwUdJazV3AdyV9G/hdRPy5yjL7kh508tfU5QuDgFsq5k+veP1BN5/z+4hYAayQ9BSwfY7Yyn3g3AXcHVmfTpIWkjpOex5YFBF/zZb7JfA54BpSwvhDFu8AUncHZTMw2whOBNZSIuJ+SfuQ+p/5lqTrIuLMLouJ9KCYad1tppvxSisqxleT/pdWsW4pe7Nu1lnTZf01rP1f7Pp55a6W746I/bqJZVk3081ycdWQtRRJOwDLI+KXwHdJjxcEeJH0qFFIT+p6e7n+X9Lmknaq2MzUitfKkkJvngRGStpO0mBS1899NU5S+YA/DfgLqZO1EeXpkjaV9KYN2LZZVS4RWKvZHThL0hpST5GfzqafB1wt6fGsneAjwPTsgA2pzeD+bHywpBLpRKm7UsN6IuJVSWeSnmD3EHDvBsQ/H/iwpJ+Qejf9n4hYKelY4EeStib9355N6uHSbKO591GzCpIeJnUf/XTRsZjVi6uGzMzanEsEZmZtziUCM7M250RgZtbmnAjMzNqcE4GZWZtzIjAza3P/HwzgR415a9TCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def eval_show(steps_eval):\n",
    "    plt.xlabel(\"step number\")\n",
    "    plt.ylabel(\"Model accuracy\")\n",
    "    plt.title(\"Model accuracy variation chart\")\n",
    "    plt.plot(steps_eval[\"step\"], steps_eval[\"acc\"], \"red\")\n",
    "    plt.show()\n",
    "\n",
    "eval_show(steps_eval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从图中可以看出训练得到的模型精度变化分为三个阶段：1、缓慢上升，2、迅速上升，3、缓慢上升趋近于不到1的某个值时附近振荡，说明随着训练数据的增加，会对模型精度有着正相关的影响，但是随着精度到达一定程度，训练收益会降低。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型预测应用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们尝试使用生成的模型应用到分类预测单个或者单组图片数据上，具体步骤如下："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 需要将要测试的数据转换成适应LeNet5的数据类型。\n",
    "2. 提取出`image`的数据。\n",
    "3. 使用函数`model.predict`预测`image`对应的数字。需要说明的是`predict`返回的是`image`对应0-9的概率值。\n",
    "4. 调用`plot_pie`将预测的各数字的概率显示出来。负概率的数字会被去掉。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "载入要测试的数据集并调用`create_dataset`转换成符合格式要求的数据集，并选取其中一组32张图片进行预测。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:59.801973Z",
     "start_time": "2020-09-04T06:46:58.822022Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Row 2, column 4 is incorrectly identified as 2, the correct value should be 4 \n",
      "\n",
      "Row 4, column 4 is incorrectly identified as 3, the correct value should be 7 \n",
      "\n",
      "[0 1 2 5 3 7 0 8 3 3 9 2 8 0 2 6 8 5 0 2 9 4 3 3 6 0 2 3 3 4 7 6] <--Predicted figures\n",
      "[0 1 2 5 3 7 0 8 3 3 9 4 8 0 2 6 8 5 0 2 9 4 3 3 6 0 2 7 3 4 7 6] <--The right number\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADsCAYAAADXaXXTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOy9e3wU1f3//zy7CbkQCAQSINxCgIQolIsRioKgfgHFS+EjUBVLRSkKUv0oqPWDn4oWaq0Kv1pERW5FKBa88Kloi7YaiqBYRFRKJEAIt3ANCZCQhGT3/ftjZje7yW6y19lF5/l47GN3Zs7MvObMmfeeOef9PkeJCCYmJiYmxmCJtAATExOTHxKm0TUxMTExENPompiYmBiIaXRNTExMDMQ0uiYmJiYGYhpdExMTEwMxja6JiYmJgUSF0VWKFKV4VykqlOKgUtwZaU3eUIoZSrFdKaqVYkWk9XhDKeKUYqmen+eV4iuluDHSuryhFHlKUaUU5fpnT6Q1eUIpVinFMaU4pxQFSjEl0pq84ZKXjo9NKf4YaV2euFRsgFJkKMUHSlGqFMeVYqFSxPhzjJAbXX8F6LwMXATaAROBV5Ti8pAK80CAWouBucCyEMtplAC0xgCHgWFAMvC/wFqlyAixtAYEmK8AM0RI0j/ZIRXlgQB1PgtkiNASuBWYqxRXhFZZQwLR6pKXSWjPViWwLuTi6nGp2IAAdS4CTgIdgH5oz9d0fw7gs9FViiKleEIpdutWfrlSxCvFcKU4ohSPK8VxYLme/mal2KkUZUqxVSl+5OW4zYHbgP8VoVyET4G/Aj/z50KM0AogwjsirAdKAtVnhFYRKkSYI0KRCHYRNgAHIHADEc58DSVhvv//EaHasah/ukej1nqMQzMWm6NNa6htQJjztBuwVoQqEY4Dfwc//xxExKcPSBHILpDOICkgW0DmggwHqQV5DiQOJAFkAMhJkEEgVpCf6/vH6cdaBLJI/90fpLLeuWaBvOerNqO01jvHXJAVgWo0Uqu+rR1IFUivaNQKkgdyCuS0ftzh0ajTZd0FEAHZAZIUrVpdzvMxyJxoLKuE2AaEuZzeD7ISJBGko36esX7p8/NC7ndZHg2yX7+QiyDxLtteAflNvf33gAzzcNyhIMfrrfsFSF6QhSPkWuulCaXRDbfWWJB/gLwWrVr1Qt9Cfxh+DnIepHu06XRJYwUZAvIkSGw05qlLmi4gNpBu0Xj/Q20DwlxOc0C+RDPeArICRPmjz9823cMuvw8C6frvUyJUuWzrCszUq+tlSlEGdHZJ70o50LLeupbAeT+1GaE1XIRNq1JYgDfQ2stmRKtWEbaJcF6EahH+BGwBRkebThe9NtFegzsB04LQGXatwCTgUxEOBKkzXFrDYQNCrlN/ljYC7wDNgbZAa+A5f4T525Dc2eV3F7ROJYD6Q5UdBuaJMM+HYxYAMUrRU4S9+rq+wH/81FafcGgNF2HRqhQKWIrWOTFahJpghWJcvgqgAtwXjNMZQxBtujrh1joJ+F2A2upzqdiAcOhM0Y+7ULR2/WqlWI7Wsf6Yz8r8rLJ/C9IJrZ1kM8hv9Sr7kXppc0EO66+MCqQ5yE0gLbwc+02QNXq6q0HOglwe5GtQuLTGgMSDPAvyhv47Jkq1vgryOUG0ORqhFaQVyChHXoJMBKkAyY4ynWkgt4MkoTUvjNJ1/iTa8tRln6t0jV7TRIPWUNqAMOssBPmVXk5bgbwLstovfX5eyBMgu0HKQP6E1pjc4EL09DeA/FtPewxkneNCdGPwqkvaFJD1euE4BHJnCApHuLTOAS3nXD5zok0rSFddWxVIuctnYhRqTdXTndfTfg4yIkp1btLTnUN7sH8RrWVVX/cayBvBaDTouQqZDQizzn5onb6laJ2+60DS/NGntAM1jVIUAVNE+IdPO0QQU2t4uFS0Xio6wdQaDqJdZ1REpJmYmJj8UDCNromJiYmB+Ny8YGJiYmISPGZN18TExMRATKNrYmJiYiCNBkeMsIyPqraHj+zrvDrLm1oDx5vWS0UnmFqD4fug9VLRCWZN18TExMRQTKNrYmJiYiCBDjZtYjCVYwYCcPgWe5Npk/Kbkf7C1nBLMjExCQDT6LpgSUzk0EP9sMdBxrul2L/Oj7QkQDe4004BcKDPu02mf35Qd/5cNYq0habh9UTZpMGcy2zY5Gaphi5/2In9woUIqPr+YembQ9HY1h7z1TZ8AABHh8cTf5ofVFk1mxdMTExMDCSkNV25uh+lWQmNpmm7oyxqapD1US2SeOu+F8hplkhm6n1kL86JuNaLo3KJ++UxPsp5z+d9Hk3ZT/XUj9m8MD6Myi5NLvzXIIY8tI0XO+xosO1IbTm3nHuMmAtC2qZj1BYWGS+wHhdH5VKeHuvXPknFNTTbuD1MiprG0jcHgD1TW1I4dpFbvjo4d0OFlmboItaWJzO/7A5arfsKqa72eMxowJqayplR3YPWGRKjq3J7Y2sey8mHq/h64IpG03b761R6rOmvnbykEvuu70IhIeQUjn2NnJPT6fJ1ZHXIzNN+GdxLDUuLFtQO6NFgfWzxWQBsewtDer5RczbxZFvPZa5TTBJfzV4EQM7i6WQuraX28JGQnt8fVG5vOj61j1UZeX7td1fRcE6X9Ea27wqPsEaw9swkf1oLAA7c+hrgnq+emJB0livnvcDdZx+h2bkaYncfwXbqlCF6myKmcycALmamcionnrefeN6pE6BZ4Sm/y0hwRlcprG3bcu2Kz3g0Zb9Puxy4dbE2hypww3c3Yb0jDeyC7fRpbSRCk4A4aatw/k6zNifOUoM1tbNh+WptlQxxcX7vVzEwg02vLW6wvvvHkwHInpkGtbXYSs4ErdEb5fYqTtlq6Rab5FyXP3UROTKdLk9HwOgG8Fy5siojj7nLerH5R8a/6eT/KoUDN9bdz2qpobDG+9j5LSx2OsUk0S02yVkOBv7PNFqviKzRdZTnvQ9oY6EXTHpF35LkVl5zXvO/jARldK1t2zJ162fcmFgK+PcKBPBW1tsc3mZnT00ai68aHDX/bpciQ1bPcv4umPQKD7b+jp5bTxiWr0dXpPNmv6V+7xev7EBSg/Xbh70MwPFtMP/ECA4NClahd8YV3Eblgo4ejX8kCPa5iiZeKu3FJ9d7n1jj2LgejdaCI4WjPLexOioszUN27OBquhZFduxJ4lRig013FQ3nxBPdnMvjX93I1ORitzRJlnhymkFmbCl7Pynkn3cPjsgr0aXIgZpy7p5R95rTs/Awhfd0cW6PU7Fkx54ESzAz3vjGmQ1ZrOuzhKzY0BXM1tZE/Ru6JpRwiNDU2j69J5e7XmrPqow8+n5xBwDpTwrN2oRiJqMQ4eG5cmhNW9AwH9o9e8DvJohwULAkl79c9wqufxTV9lhsJ0563Se2ItiZjkKHtU0KHT+opmtCCROSQ1ueXQnY6Fp698L+0nk6x7g7QIzIvwWA2t+3o9mmusb8N2bfwuvJFlrddaRBG2WciuXRlP18/VJnjj6dG9FOgGhDvdiWEY/d4pZna8uTmf/M/bT6sK5B/9Csq7hn/EZDtVlbJXN0RXrIDa4rM48N4It5V5LItpAcT7bv4ujTuVyZnkO7gkoA7Lt2EpuaSv9503nv8d/TKUared8zfiMrK0YBGOr3bC87y+QnH8EWW/eH6dCqtnzVIP3Rp3MZ8ViLiLb9FyzJZfm1yxgY51/NXDzUCTytM4SYGB5pt4GcZok4arYOe1a2qhPlXRX5U+tq5Znv3kf2+lKa9pyvd5pA9dW2SeCjXm9CvRrIvqJ2AGTVM5yJ72wjEagsHUhOv+kA2HPK2TN0pTPNqow8RjzWgsqEgSSs/yJQaQFh7ZnJnqeSaW91WRmpm+9Cs43bqUyoyzNA82tctdVthr3ynIsBtf8FRVwcb/Zb6rPBzd48CUt+w6aExmhZKLR657NA1Hml2cbtpNRbZzt1ig5vKc4/WleJeDRlP4tyLob03L4g1dUkr/rc5/Sn+zRjXFuD73097h34KcMT3M3P82e68/bi60jD+x9W2x1ldPvrVK2vJ4J4ev6v+XYsvJIKQOtTlVz8ibvHQvdexVR060CCn53tIXUZm3xoKG23NP5Pl7D+C7qs137L1f3oG3cHXw9c49z+Uc57XDNtLJUMdKY3gpr0ZPZftxxo2FQSaVzzzBNlkwYz/PLIN8tMPjSUvP9ke93ec1ktakt0OsHHdO7E3gc6u7Th+Vaeo4E2I4t5KnV3RM7tCCgamPi6c93Tpy4D4K03hpPeRNCD/et8zZvp1rDKbBJvz/+pfpqJtOdUscfFToFmq3L6TW/02fSEGRxhYmJiYiAhrel+/rc+dFnme01GbdlJ+tle3PDSTbyV9TZJFq2p4l993qXbLVMAyPLzXyRUzDw2gJaFl4YLW8Jdx1jeZXOkZbD1YDd6LqsFtHsb7Tic+E8PaEV5V0XBpEW49lJvPdiNrnsqI6TONy781yAGtvl3xM7vGlDkYMX2qwDI8qEdPCYzg8JrGw+oihT/6vMu9HFfV26vYlzBbVTVam9AgdiIiI+9YN/1HdbxKTz8wfXMS/+QNKtW6Fu20fxOLb17RSSAYvMfB9F6ZWjbEgPB0rsXtW3qCmX9gBKV25tOSZFz4Hdlz9CV9I3Te9lj+mOtqIlabxR3J/5XPKZxvZ70s5Eph/WxpqZSc1kn5/LP5r3XwCsIIC32HPZhg93WBeLI3xiWFi2oGJihu/0FxqlrOrh1TgGoKK3rnLRVMLt4JJbxlcSVHAUgjiK/jxNxowtgKznDoUGw+JsrnNFCjnbe4fPHEDcyvOdXcXFcbBlFbXe6czwWhf2l83qHpYYzoETHmwN9tdSwp0YLPDESZ/v8Gq0j5ZNrM6Mu8MXaKpk9TyVz4LqmO28c13PDSzdhHZ8S1iANT9QPOvHVr3VqcjFT1yx3WxeII39j1A7oofs213WOnrRVoKqs3neKIiyJWu1ctUiiysPzf9JWQYmtrjd9/omRHBpUAVQ0SOsPUWF0I03Z+P6smfcCnpz0I4HDOT479qTuklfnIeIIKHGQGRuLJwf6l0p71Rm8CGF0gIavHF2RzvYrXsafTtO3st7m4Q+uD2uQhifqB520sHgOJokWhqyeRfY87e0m8PqvMRx6qB8Ab933gscgnSGrZ9FjgUuFpraWYA0uRJnRdXVcNxJ7jHILAY0kKrc3167QopE8BZ04Akqa4ufJ31CwoT1HR7cOW+3MXnKGhyZOR2KURwf9OBXLjYla4Eu1PZaNc4aR+E5o/G2DoXVipTP4whMDnpnGZZPy3a4nyRLPvPQPmb1tJEdHx4UtT1Vub4Ysq3O3DIWT/oBnptEmv4rMwkPUBiuwCayVCvv582E+S3AULMnl3oGfOj0uXNujXVk4bgm/tGl9SxmzQ9fUGFVGV7bv4kh5hvHnreePm7N4Opn/Kg57Aa3PxVG5dHxqn95cEFxzR5q1OQs6/pNx626jujYDgMpVHWgVwnZqqa1FbdmJAmfAQd3oUZr/tSPwBaB0diJf3p9ByYfpgLEBB66oF9tyZfo0r9vbb9jH0QM9GgQcpFmb80i7j5gZMyZs2mzNY+sNyOO7wXV15Hel/YZ92E6dMrw8e6N41lW0Gam1Qw9t4/4nHK5nzxHI0zqxkuXdlzXwKQZ9lLnnHuOJh1YzIeksIxNruNg+9JGKITW6P77xW3Yd1hrvU5ZFvhMqUDp9UhmRYf3K02NDWstPssTz917vO5dzMqfTKmRHd8cRcJB8QBucOid/OlXta92c3l/ssAM67ODpdrofJ8MjYng9BUe4YgOabTxFZcJArpk2FtB7soH2VtjzYkeyn04K+Qho3rjm27Gc+qyDc7l+vjrSOBz5U9a7P3u28Eusw4eAovKci3zb290t6aStgiGrZ9Fz6aGQdvbJ1f3Ye08Mljgb26942e0NZ/KhoQB8/nfNRUEUVPerISP2NOEc8yJgoxtbfJbuH09m+7C6C1neZTOTp2rbd9cO9rlW5Tpjwz3pxoay/pBIHXzMOe1PuIJOrHnaOLVd8jQPgW6x2uvZX657xRki6nDkT7y7+pKb4aK1NZH91y1nxOuTsewN/fFji8/S7W9T3NZ1fs9Cl/VaHln65vCd7nXhyqnPOjjTRJKmypi3QJ4Sm6LHgv3UNjJOgz84ZqY4PK2WA0OX6Gs1OzUi/xb2FbVzBr50WbbVGRyz7Yb/z+lBFS7M4AgTExMTAwm4pmvbW0j2zDSOb9NGgnLgcNKf+VAF/7IOxlojXkdaj8nM4OSwDtQmKjZOqxtoBGD1+TYAHN3RgcwAfOF8weEcX5IbHf2sScU13FU0PKAmhufPdOf9Yu016Yo2hzzOjGB00IltbyFZ92qv4D9dMo3l17q3pT2asp+uD642fNYAx2wMrZ2DyLgHclz4r0FUeRmcyeEcH1NSGZbeedc888TpAa0a+BVHUyCPpzKm4uIoG98fW6zimhkNZ+0oqKng9p1T6Fjd0N84UI4O0zx+9gx1d6+7q2g4tb9vR8/yWkqztJrumcmDPQbHPH+mO0n5PvRa+0lwbbq1tcw/McItqMHBix12wLwd2hCELiOtu1J4bYKLY7S798CTn2ptaVm/Cl/bcNGY1pqOsZqGaqnhpdJemlN/2M7qnWYbt3Oiqj+syfNrv7Xlyfz51brX9C/+axBz59RNAji19ZekWZvz4YVYmh2PjD9y1pTtTF5yD3+5/hW3kagmJJ2lz9z5zPxoTKNDAIYSmXmaf/de73yNz9riPoOFt4ADgMO1dix31GA7YWyghGMGg/KuDRtNoyWQx4EjsMk+TJsh5mLLWNbMe8Gjh1BBTQUTv51M+zH5hrQ9716ZQ5uqKk48Wu11lpvFZ9M5WdOStxdf1+TYEYEQlNF1BDXM3jaSBR3/6QzjdcV1RHhfOFJbznm7JSIO1oU1NXxyfXfkRHRGUXniQE0585+5n7RVdYUj8Z1tbH6n7l4c3DaSR9p9xMNLZ5HxbOgLkbVNCsTUK0rV1djLK7C0qeuyypl9iDvip+oDi+jJIhTEAWCJ0x5za7s0rzNYuBIprZYWLVxmMHCvuR2oKcdSGx21XAeuATJ1uBvcUptWKRj/1RTajzFuHsIdv24YfVhur+Jwbd17y1tTRqK27Gx0dLRgCIn3wtHRcYxbd5tbT3mg3PLcY3R4ax/ZF7TOluh48Y9e7p7xiDaubiNpjo6OY2bMGLqc3xmW/Oz4QTWPtNvgtu72nffS7vk4Fqx2NxLa0Hl1PciRDOJwnZ3C2wwWrkRK657fXcbnt76gL7m/Ufpy/6OR3E0PAJA9o9BY7woPjCu4DcsddW/ilpJdYc3PkBhdW8kZLA/2YkSbyc51R9yaDhrHdRaEDrv3RlX0ktHE7NjHsPumsmLh/EYDNhx5lpiXj72JttBwh652TShp4GC+rv8S8pb09Op47qDaHhux++06O4U3XGdssFbUIKeMfwuSeJvXHvVm52oiNoOur2W1Pt3+OpWcF7U/LlvZ2bBoy1x2CIAReZObSKmNZ2Jkc1HI/HTtu75zc4XIPJzBlQe9O6C7Yq0R5ywIRv7rZawvBSAz7T4Kx75m4Jm9Yz9/noQPv+aO2bPcZg6ojyPPmjK4kSIrtjlZXtpFHYR6VohQkvnufbTZbnGbsSGaapMOR/4Ou/dGrKboWlYf+fUaJiQ1bkBzFk8n6aDQa0dZ2H2cHb6+Fh98fo1+mw5bRFptYREpfgQYRKJA27/W2pKyF+eQc2o6lmrocj7yQxL6OnNAtBiBtxdfx+q21/HjG75tdIjJ+jNHhGNWCF+oXNWBnMzpjabJfrfUWT6ikfN2Cx3e2hfxt0JHWX221USeatt42kw98OGH3mQYVWHAkcL+dT5d9Ck3fugFIhAcXhO7Dg2m29XRP3NEq5WfNRmZF03loO2WWLrhHjChqqzOfo9owJcAl2gJQ440ZnCEiYmJiYGYNV2TkJGy7DNSlkVaxfcPb/kaTbVxE98xa7omJiYmBmIaXRMTExMDURJF06iYmJiYfN8xa7omJiYmBmIaXRMTExMDMY2uiYmJiYGYRtfExMTEQEyja2JiYmIgUWF0lWKVUhxTinNKUaBUvZjHKOIS05qjFB8rxVml2KcUYyOtySNKxaHUUpQ6iFLnUeorlLox0rI8oRQZSvGBUpQqxXGlWKhUdAYZKUWKUryrFBVKcVAp7oy0Jk8oRZxSLNU1nleKr5QiKu+/A6W4XSny9bzdrxRDfd5ZREL6AYkJYJ/LQeL0371AjoNcEWptPyStIDEgBSCPgFhBrgOpAMkKt1YB//IVmgvMEcgQsAjcLHBeICOa8lTf5wOQFSDxIO1BvgV5MNruv77PGpC/gCSBDAE5C3J5tGkFaQ4yByQDxAJyM8h5kLDe/yDydQTIQZAf63o7gnT0eX8/TlQE8gTIbpBSkOV6wRsOcgTkcd0AvaGnvxlkJ0gZyFaQH/l4nmyQYyATgsjIH7xWkN4g5SDKZd2HIL8JuJBCkcATArsFSgWWC8QLDBc4IvC4wHGBN/T0NwvsFCgT2CrgU77q+34jcFs05ameNh9ktMvy8yCvReH9bw5y0fVPFuQNkN9Fm1Yv5/oGJKD7b0AZ2Apyb8Da/LyIXSCdQVJAtoDM1S+iFuQ5kDiQBJABICdBBqHVsn6u7++oIS4CWVTv+ItALoAIyA6QpCAz/AetFaQPDY3uRyDvBqpVN7q7BDoLpAhsEZirG91agecE4gQSBAYInBQYJGAV+Lm+f5x+rEUCi7ycp51AlUCvaMpTffl+kJUgiWg1nF0gY6Pw/vcHqax3rlkg70WbVg/naQdSBRLQ/Q9zvlrR/sx+BbIPzYAvBEnwWZufF3G/y/JokP36RVwEiXfZ9gr1alQge0CGNXEOK9pr0JMgsUFm+A9aK0gsSCHIY/rvkfrxNgaqVTea97ssjxbYrxvdiwLxLtteEfhNvf33CDSarwKxAv8QCLb2GJb7D5ID8qX+4ApaU4OKNq0gQ0GO11v3C5C8aNPqodz+gyDeHsKcr+n6fd8O0gGkLZpBn+erNn870g67/D4IpOu/T4lQ5bKtKzBTKcocH6CzS3qPiGAT4VOgE+DbtBOmVo9aRagBxgA3AceBmcBaoOmh9APUikgDrShV5vw0la9KWYA3gIvAjHDpDDRPlcICbATeQZusrC3QGngu2rQC5UDLeutaAuejUCvgzN9Q3f9waa3Uv/8owjERTgPzgdG+ivK317Wzy+8ugGM+FqmX7jAwT4R5fh7fVVf3APd18IPXKsI3wDDHslJsBf4UhE7wUysivuWrUgpYCrQDRiNS08QeIdXpY56m6MddKEI1UK0Uy4G5wGNRprUAiFGKniLs1df1Bf4ThE4IU1lVCrf7r1cagiXkWkUoVYojHo7hO35W178F6YTWRrIZ5Ld6df1IvbS5IIfR2kgUWqP+TSAtPBw3DeR2tB5WK8gotF72nwT5avGD1qqn/xFa50EiWnveAfR2qoA+WvPCtwKd9DbdzQK/FUdHmnvaXIHDepuuEs1D4SYBj1oFXhX4XCDg9nGD8rQQrT0vBqQVyLsgq6NU65toHgzNQa4mSO+FMGt9FeRzgugfMVDrMyD/1u1Ba/3YPndQ+3sRjt7AMpA/6Q9zg4vQ09+gCytD6+Ff57gIPYNf1X+ngmzS053TM+oXIcjwH7RWffl5tJ7bcpC/gfQIqjC7ey+UCfxJINGj0dXS3yDwbz3tMYF1TqOrGdlX9d9dBUS0zrNyl8/EKMzTfiB5er6e1tOmRen9TwFZj1YxOARyZzSWVZCu2u2XKr2sOj4B3X8D8jUWrXOtDM0D4iVc2oib+vg8tKNSFAFTRPiHTztEEFNrmFCqCJiCSFRrvZTy1NQaHqJZa1REpJmYmJj8UDCNromJiYmBmDNHmJiYmBiIWdM1MTExMRDT6JqYmJgYSKPBESMs46Oq7eEj+zrlbZupNXC8ab1UdIKpNRi+D1ovFZ1g1nRNTExMDCUqB182+X5SPOsqynMuNlivqqxk/2o39vPBDgtgYtI09cthUn4z0l/Yatj5Q2p0K8cM5FQ/z4fMXHqI2sPBjrUSHix9cyga29rjttSdtQAkrP/CSEnfO07OuIpJd2/k0ZT9DbblX7zAzGfGgGl0fcaSmMihh/phj6tbF41lNaZzJwrv7dJkOks1dPnDTuwXLoRdU5uRxXzbe71zeXjnMfBC2E/rxGxeMDExMTGQkNV0L47KJe6Xx8jPec/j9hymk7kyhtrColCdMmgsfXMA2DO1JYVjF3lMMyL/FgDOtB5M/Fk7ie9sM0zf9wEVF0fZ+P488eBqJiSdjbQcn7k4Kpfy9NhG06RtOhaR8mxNTeXYhJ5snPZ7OsUkOdc7yurFylyabdxuuC5PXMxMJX+q52fLlSO15dxy7jE6rN2L7dQpA5RFjpAZXZl5mo9cDG5BTQV5F3oyNVkbTS1/6iIGFk2jdRQZ3aIxWpOCN4ML1F3TPJh7uheb34k3QloDYjp3ArRCXJ/Y3UeisqBaWrTgwvAc1sx7gW6xSU3vEGFUTAz2Qb2RGEXHp/axKiOv0fQ5i6eTubTW0GYza2oqx2/rwVezFwHueeooq3c9NZzTJb2R7bsM0+UJa2oqp3J8e146xSTx1exFjNg1Gcum6CvLoSTkHWmlNq1NZvxXU0hbEM/UNctDfYqwYRM7+2sr6RrTjDjVeC3HSCwtWrD3AW1o0IJJrzTY3n/edNovrzCkPcwb1lbJEBeHXNDGeJaLF7kwPIdNry2mvnEwEhUXh6VVsts6e9lZpLq6QVpLmxQWrF5ETrNEn46dP3UROTKdLk8bZ3RP39SDHb9uWAZcWZWRx9xlvdj8o8hUEBz4ovWHSMiNbu6mBwDInlFITd/MUB8+rOyvreS/b5zMtWu/9NjhEyn2/O4yPr/V0dLfvMH29x7/PaNaPkanZ43rga3P0RXpvNlvKTe9/zAAaZ8p1sx7gUgaXICy8f1ZPne+27rJTz5C8qrPI6TI5IdOyI2uvdoKgK3sLDE79jHsvqmsWDifbrFJzPqfP/Ns0kTSFkbOOLiSuewQADlqOvlTF9E1phnXrv2Snyd/gyfjFikk3kaa1bueTjFJLLj3dX6ZNEhMT1kAACAASURBVAWAjNmfGSUNgDMbsljXZwlZsc15c/RCAIpGtI2KJgV7jGpQc7XHePZbt5ec4aGJ05EYRbtnD7g1L9xVNJzdK3Ma1Nz+8LPX+WWz8Of7yRlXAfDEg6vd1g94Zhpt8qs4+XAVXw9c41w/tfWXHNw2kqOj47CVnAmbLn+0gtYk0+mTSufykWsTfGrzDRVnNmSxOns50Jy+X9wBQPqTgt0wBWH207WfP0/zL4qoEs1JYkLSWZ5qE84z+oejLS5zZQxXHpxGbaLivcd/T5q1obGYeWwAX8y7kkSM7Ugr/N1g5g5Z22D9kdpybnnuMZ54SOugGplYw6/HaenmWCeQ+avwG15rq2SOrkh3GlyAgXGx+rdvnWZry5OZ/8z9tCr7KiwaxYN9HfrLbXx5ZwYlH2pTYDl8NKW2FrVlJwo4+nQuV6bnOPdJKq6hTVVVg2ONTKzhYvtQzCzjneJZmrsd4OyMdNz/Dm9rHU/tavuR/eAkAPYMXUmatTmPtPuImTFjwqrNE1X6M16/4zRn8XQyVxa7dT4mZQ42UBn8pMs3zrJ6rkT7br/Lc6dj8ayraDOy2G1d6YUEOt5djK0s8E7hsBpda89M9jyVTHury0qvwXGRo7awiFSrhT1PJdNcefai+7Kki+GeC0XzBjNn7Fomtihxrnv+THeWvTVK82tcvpNn1UQO3q/5vzrTjV3LM7YJ4a/xxsXxZr+lzkLcGN3+OpXpQ/7ZoNmmoKoDyas+D2LCqcZpu6OMbn+dyoFbFzvXvdhhB3TYwdPtLgPgLYY3cI5vtnE7KfWOZR/WP0wqG6c852KDfDtvt9DhrX3ODlS1ZSeWEVoNk6FGK3Sn4ybtzyn7sknsGbqSk7YKhqyeRU8XX/3KMQMBaHVXdPruF8+6inE/y+Op1N1u60ttF8hd+ADZT5/FtrcwoGOH1ejWpCez/7rlgPZ6NyL/FqcDd7RRXyvA5ENDyftPNqBHrVBkqKaf3fKJm8EFeL+4D13maAbC2yvRxBYlHLjlEzbPjmxHiis91tTwfmYfw9vK7V/nk/NiJt2TJrN92Mu0ttbdX8cDlXh3NX+uGtVos5elbw4FdxjfuVo2aTDDL3f3QviiuobbP3iY7AvuBsHxbI3Iv4WPct6jjVXY93B3er7czFgPi7wdAHSt6Ue38ilaxOG8XdS6BL84gqi8uZhGksYCeVpbE9l/3XJGvD4Zy14PO/uAGRxhYmJiYiCGjr2w/7t02rS2UDlZa8dJKq6JGiduT3z+tz5kPR0dnX6rz2sNZQePtqHt5A7O9ZEKOrCmpnJsXA9aWBrvgii3VzGu4DZiSiop2tGB1V3bNKi9hxvb3kKyZyQzfMW9rOu/pEFzyKMp++n64Grml91Bq3VfOd3J5Op+lGYlAFCSa+fAra81OPbzZ7qTlN8sbNoT7jrG8i6b3dZ9eL4PPR/Y1uBNxxH+e6b1YJgHadbmFEx6hRF/m4wlAiH4astOsrZov121ytX9sOeUu6V1LSdGdmp54rapHztruY7nbkd5V61ZKgQYanQLx74GY+uW7yoazomq/qhawbJtF1IbuaYHa0UNz5/pzoOtv3P66FZ1qHW248WUVGLf9V3E9P0+fyQA8Qfj+Pc843p7vVFzWSePDvqunLRVMLt4JJbxldhKjpL5K3gydSwTb1xinFAdW9lZ2o85y8QNk1ndZ3kDwzsh6SxXznuBu88+QmJePnTtSPGj1Xw9cIXXY64tT+bPr44iPUq8cRyoqBrksCF774nhwFD3MnC41o7ljhpsJyL3jHniyU81g5W6ORZ+G2VGt/RCAvkXL6Cq6nrNVK2Qf9HdYb+9FWe72qqMPFiTpw14MmgMthMnQyXHb2T7Lj65NpOeW09wY2IpcSpW63y5Vds+fNcY4kZGTF6dO9DAyGnwl8WlV3BoUAVQEWkpTlJuLmD8+inkXbHUrX0XoFtsEpteW8yw+6aS8PBRvu71fqPHeuG3d5K2InwG19omhfgYd8+IcnsVByvbEE156o36gSn2kjNYkppjibO5pauWGvbUpIE9yv8tQkTIjG7Hu4uZGTfG2bhvByzbdjFzkLvLyp4XO+odVtGH7fRpFl81mL2fFEZVcIRJaOl4dzG5Cx/wWg5XLJxPqjUGiGxHZMcPqlnQ8QM3HeMKbsMyvpJLwejWD0x5aOJ0jj1azfYrXsa1w/ql0l58cm0mttOnI6DSeEJmdD35rUltbYPaa/bTSXQrnwrg5sYTFYhgO3WKf949mK9f6uzmHL84ezUTN0wm5eYCw+R8ek8ud73U3qcxAJr1K3Vzjo80dxUN5/SDnYDIxv97wlZ2luynz3oth96COhafTWfd/aOcy21378PmMWVo6JpQQpJFM7iujvy2kqNhPGtgnNmQxU+6fOO2Lit+jVtgyrglHzI8cS+t6wX6VNtjo2LsEGubFDp+UM3U1lsIZ3CU4YOY2/YW0usVbRDQTNt9WjtvlCHbd3GkPMNtXVZsc37S5Rs2G1j7ke27OPp0LiMea+E2mNDa8mSef/ZO53LmpmJ2d25rmK7GcIx0Vfv7djTb7t5J6i3Qw0HBkly6dtFqO/UDF0LN4Z+0Z/qQjX7tc7KmJZZNdUEc4TS49WnKkd8VTwEh4cJTgIw3tMGv3NNEKujIIzExPNJug1v059wh7wKwo1/X0J0mZEfyg4puLQDo3qu4iZQmzTZupzJhIDn9pjvXxZ+GtOV1xujQrKu4OzcvAuoaMqSt1iyzYkI7ml0z2C1Ao+OAYx49FywtWrDnd5fxl+sXOiPaGgtcCIaieYO52L6Gu3PzmmxCuubbsZz6rM5TJP40pBH+TjNHfvymxULAd99gowMOHMFP2694uUHt1VcGJB1kwxWDyPxIu2aJt9FzmRYZGA04yqvj2xnoUXiYQLv9Qz5zhIPGRq+PpGP0yRlXUdUWOuZVOZ24PaW5M92/WlA4SVj/BV3We9/eZmRxg8iZSOHQ8dSNu/nwQiwP1fzCue0eD3k6MHE/K2dfy+e3vuBWw3Ac55ORWUGP6u86e8Eff/o6IxN9C9s99VkHZyCKkajEBN6/aYHPo505qP9cldoukLvpAbKLz4a8Vm7pm0P+tBYcuG4x3gKKemSccHtD88TEFiWk/nQJM6xTnGXgyk+nkbIlxII9MPzyPQDsnjSYVit9i94ssSl6LNhPbRCd/mZwhImJiYmBhHzmiH1F7QDIaqRmFrZAey9YU1M5dXMPoC6YIEem0yXPPV1jsxzkVVpY+sUQsoiuYI4L/zWIgW3+7bYuWrSOTKxpcgSpkYk1+hjB4em4iMnMoHBSemAjWUXKg6m6mtt3ugdy9Mg4AWjPmaeAIk8BB8dtkD3zaFhcMU8PaMWBW91HXJt5bACf5vWmrVaBZH9VOuR42Lke4S4D3nAEncx8qIJ/WQdTm6gaDfYpqKng9p1T6FgdXLNoyGeO6Hte62W1D+uPtaKmwej11p6ZVHUwNgii5rJO/Hue98GUramp1FzWiYstY73OcjBn/61kTYkugwswas4mnmzr7lAerVr94YtqrQng4KG2ZAUx5sWpazp4NbiOc3x4vg9pseecs5xEGkcgx9pvrnDeW2+zQqjc3tiax3Ly4Sr2uHivnLRVMP/ESAhTwJGnAIwP3vsxif1K+fckz140i8+mc7Kmpdu6rPhjbhWcxWfTiT9rbEzaix12wDxHU6Nnz5WCmgomfjuZ9mPyg26qCXlHmtNtaY0WIvnJ9d2BOsfoPU8l6+1AGpFyjLYlCNZ2aQAcG+eY/gRcM/2krYISm9YVXHohgfaGKoxuPAW+NEZmbKzH2TiqpYbCmoZtrLd/oA2GnvVA+Hq179iiuYxlzzxKxcAMpr4WZS6MHliVkcfzK+qeq2tXfOaxQzASgSne/twcM7K8NWVkgw6yDXfdQR8XX963powkcUv47vnByjaU26ucrnhN4ToTTvsx+SHREFbvhQdbf8fN274Fos8x+tOJL1Byu2ZQtVeKhv9wQ1bPoscCrUB3rC421EUo2vEU+NIY1/5zv0fj8FJpL6cBccU1yCZcbB/2MgDHt0G88lwGohHX5yozNhZ/PBwigWNGFkvBrgYtNq3WfcXMj+rKkaWkYZpQcnR0HOPW3cbfm4g2dOA6E06onv+wGt04FUtOM61AeHKMdjjQy6nwOtDXn8ECtMFA0qze98lZPF0b/zOCocmN4cmR28iR8D0FvjRGtb2hYXDe/xORCaBwhAG3bqQcRBJvwTGuz5UrjvuftiAea0UN0RCYsvhsOm9NGakZXA9NHVJdbWj4v63kDJYHe9F37h1eg4kcs3EAZBdrTR/BDFpen5AZXfViW65Mn8a5G7TXmT1DV7pt9+QYfaS8FXEGzFhafwaLpnCOcB+BkZl8xoMjtz8O9Eazcc4w/i/5Wrd1ScU1DQIoQk3qv45x5ey6WUFcpyyPdrwFx3jDef83bQ97H6AvARjOwIct2yLWJ+kJ+67vaPd8P67MmuZxe/sNdYPDh+PtNmRG1zHSfvKBAQDk5E9vfAegZaEQZ9DA4HK+nHGvzcIe13TaTJcR7k1CQ+I72/DP6zQ01BYWkVJYhCUxkVEtH3Pe/x/f8G2DIRMddPvrVOKPxzhnQIgknoJjvNHZwAkC2u4oI2dx45paFgqt3jF2vj5fUVt2evUFDnczYsibFxwBB/XdsSKN/cIFn2fLjc65LRpn8qGhtN0S3W17kaT+/d91aDDdrs72mDbnxdMBT8USDpoKjokE9q/z6fJ1pFVcmpjBESYmJiYGEpGxF0xCgO5A3zpRm866clUHUnwMZTSBlGWfkbLM8zbTS8UknJhG9xLF4UDvwKi2cRMTk+AwmxdMTExMDESJRJMzh4mJicn3G7Oma2JiYmIgptE1MTExMRDT6JqYmJgYiGl0TUxMTAzENLomJiYmBmIaXRMTExMDiQqjqxQZSvGBUpQqxXGlWKhUdAZuKEWeUlQpRbn+2RNpTd5QihSleFcpKpTioFLc2fRexqMUcUqxVNd4Xim+UoobI63LE0qRoxQfK8VZpdinFGMjrakplKKnXmZXRVqLN5RilVIcU4pzSlGgFFMirckbwWoNudEN0FguAk4CHYB+wDCg6WGVgiQIwz5DhCT943nUlBAToNaXgYtAO2Ai8IpSXB5SYR4IQGsMcBjtvicD/wusVYqMEEtzw1+devr/AzYAKcBUYJVSZIVBnqdzB8rLwL+bTBUiAtT6LJAhQkvgVmCuUlwRWmUNiYRWn42uUhQpxRNKsVuvkS5XinilGK4UR5TicaU4DizX09+sFDuVokwptirFjxo5fDdgrQhVIhwH/g6BG4cwaw0p4dKqFM2B24D/FaFchE+BvwI/izatIlSIMEeEIhHsImwADkBgD10Y738vIB1YIIJNhI+BLURhnroc/3agDPhnoBqN0CrCf0Sodizqn4ZTinwftIqITx+QIpBdIJ1BUkC2gMwFGQ5SC/IcSBxIAsgAkJMgg0CsID/X94/Tj7UIZJHLse8HWQmSCNJRP89YX7UZrDUP5BTIaf24wwPVGU6tIP1BKuudaxbIe9Gm1cN52oFUgfSKJp0gfUDKQZTLuT4CeTca8xSkJUiBfuw5IKuisay6HH8RyAUQAdkBkvR91OrvRdzvsjwaZL9+ERdB4l22vQLym3r77wEZ5uXYOSBf6pkhICtcC3aAGR4urYNAWug37Ocg50G6R5tWkKEgx+ut+wVIXrRprZcmFuQfIK9Fm05dWyHIY/rvkfrxNkabVn3bH0Ae13+HyuiG+/5bQYaAPAkS+33U6m+b7mGX3wfRXrUATongOsx+V2CmXlUvU4oyoLNLeidKYQE2Au+gzefTFmgNPOentrBrBRBhmwjnRagW4U9or5ejo1BrOdCy3rqWwPko1Ao4y8IbaO3QM6JNpwg1wBjgJuA4MBNYCwQ7zUg4nqt+wP8DFgSpLexaXRGt2eZToBPgeT6dS1yrv43InV1+dwGKHeevl+4wME+EeT4cM0U/7kLR2kmqlWI5MBd4zE994dbqCQF8mDGqUcKhtQCIUYqeIuzV1/UF/hOU0jDlq1IoYClap99o3cBFnU4RvkHr8ANAKbYCfwpCJ4RH63AgAziktNKZBFiV4jIRBkSZVk/EEESbrk50avWzuv4tSCe0NpLNIL/Vq+tH6qXNBTmsv4orkOYgN4G08HLsQpBfgcSAtAJ5F2R1kK8WIdeqaxsFEq9rnQhSAZIdbVr19G+CrNHTXQ1yFuTyKNX6KsjnBNGOZ5DOH+n3PxGtjfwAettfNGnV9bV3+bwA8hZIahRqTQO5HSQJ7ZV9lP5c/eT7qNXfi3gCZDdIGcif9Bvb4CL09DeA/FtPewxkneMi9AfsVZe0/dA6qErROqjWgaQFmeEh1wqSqqc7r6f9HGREoDoNyNcUkPV6oTgEcmc0agXpCiJonWflLp+J0aRTX35eL6flIH8D6RGNeephvzmEpk03XM/VJj3dOTRj+Yvvq1afx9NViiJgigj/8GmHCGJqDQ+XitZLRSeYWsNFNGuNiog0ExMTkx8KptE1MTExMRBzuh4TExMTAzFruiYmJiYG0qif7gjL+KiqBn9kX+fVH9bUGjjetF4qOsHUGgzfB62Xik4wa7omJiYmhmIaXRMTExMDMY2uiYmJiYGEbXYGS98cisa29mufzKWHqD0c7NghJiYmJv4R07kThfd2cVvXMa8Ka96OkJ/LrOmamJiYGEjIa7qWvjkA7JnaksKxi/zaN4fpZK6MobawKNSyLkku/NcgqpL9/19MKq6h2cbtYVDkH9bUVE7d3KPB+tYFlQCoLTuNlnRJoOLiKBvfH1usIm3TMZ+eB7m6H6VZCUBk7r+vZdXX6zGai5mp5E91t1c5Mp0ueaE/V0iNrrVnJvnTWgBw4NbX/N4/f+oiBhZNo3UU3pRIMGrOJp5s+53f+91VNJzTJb2R7bvCoMo3rKmpHL+tBzt+/UqDbX2/uAOAtJj+WCtqIqYzpnMnQHvgvBFbfBYA295CQzQBWFols3zufHKaJTLwf3x7HvbeE8OBG7W8NvL+q5gY7IN687N57zE1ubjJ9L5ejxG43v+SnPgG26s61GIf1l9LW1KJfZf/z6LH84biINY2KRATw56nkjlw3eKAj3OgphxLbVS5212SrMrI4/kV3fnk2kxsp09rA3gZhCUxEYBjE3ry1WzPbzpfD1yj/VgDc0/3YvOPGhb4cGNp0YK9D2jDrRZMavjH4KD7x5MByJ6RjK3srCHagmVVRh5zl4U3X1VcHJZWydCmFS/9+WWyYpsDcNJWQYmtzkU1XtnpFpsUNh2B4sv9P3DrYm3aSeCG727COj4FW8mZoM8dEqPb8YNqHmm3gfZWgMSAj3P3jEdo9eFXDUYYNvGfB1t/R8+tJ1h81WBsp04Zdt5DD/UDYOO036ONmx2d7PndZXx+6wv6UnOv6bYPexmA4Svupf2YS8PoGkHZ+P4snzsfqxK6xyQ41w9ZPYseC/Y7lysGZrDptcArYuHC1/vv4K2st3n4g+s5NCj4c4fE6HZNKCGnmWdje6CmnLtnPEKzczW0e/YAqzLyvB6n2bkapLra63ajOLMhi590+cbn9G+8dy0AGbM/C5ckrwx4ZhqXTcpvkK9xKpbs2JNgCXZSC98pmjeYP/70dQA6xbgb3AHPTKNNfhUnH66qq+lGiIIlufzl+oWkWZt+2FpbtXK9rv8SJm6YTMrNBeGW58as//kzzyZNJG3hVr/2m9r6Sw5uG8nR0XEhqZ3Vxx6jGjzzOYun03PpIWpPnHSua3auY8jPHSz+3H8HSZZ4uiaUcIjg3x7C5jIGsLY8mfnP3E+rD79iz8t9mN3uX+E8nd+UTRoMQMJdx9zWr85e7nxd8oWl7YeEVJe1VTJHV6QzIXkJnv6F15Yn8/yzdwLQfsM+jh7owYjHWvBRznsh1eEvF9vXMDLRfZadI7Xl3PLcY3R4ey+2U6c4NynXbfuE5C9Zt34KHe8uNuz1vWuX0wyMi3Uuzzw2gH8tdK/CXDNjGy92qHMXyoptzk+6fMPmEDx0/jAh6SxPtfF/vzRrcx5p9xEzY8aEXhSQ+q9jXDnbfVqwzE3FTbp8inF1ACeW3r2onF83Jdry7svc7r/RhM3oPn+mO39+dRTt/+9b9szvx1+uX+j1Qk/aKhiyehY9Cw9TGy5BHjiXqZWAbb3X19tSZ+gmHxoKwOd/7+P1OJ13hlh1XBxv9lvq1fAXVHUgZblWq7YBzTaeomBCLuS4p2tvhT0vdiT76SRDO4IcfHghloeXPkaX5TuxXbgAQOf3LFzTaSwA/+rzLlmxzXmz31JmxoXHODSG497uXtSblJXubymf2gYzeXpzlnfZbLiuBkTAUDVFbWERKfU6xOo/BXJ1P04+XEUkkav7UfxoNV83eMb94/kz3Xl78XWk4d8bhyfCWtMFUIkJvH/TAq/ND19U13D7B4+QPW8XteeDnajWdyrHDCR18LEm0+X9JxuArDnBZ3YwPH3qMgBWbL+KZsdjyaDppozW1kT2X7ecEa9PxrK3yeRB03ZLLN2Y4lxudjyWjGe3YndJk7D+Cw71u0pb6OO4/w+TfWF3+AUCJ2dcxZ3pGwGXe7uyYV62WvkZecNyIRqMboC0sQr7Hu6OtaoHGe+WYv8639Dzl2Yl8PXAFc7lvl/cQTvdXTDc2IZrc28enlbLnhA0Z71f3MfvJh5vmMERJiYmJgYStpruoyn76frgap6VibSw2D2myau0MPmTaWQ9sA3PKcLDxVG5xP3ymE9toD0yTjj3iWTAwYrtWu0w617PGpLym/H8IG0W6EdT9ntME25Sln1GyrLG08jV/bDnlDuXPzzfh54G3H9HwMETD65mQtJZnj/TnaT8Zl7TXxyV67z3ESdAd540a3OnO9SVB6eR8nUINQWiZ0E8astXhpzr6DCt7X3PUP8CtIwgJEb3/w79iAnJXzZog5yQdJYJsxfhzXVozv5byZpirCFTub3p+NS+Rr0oXHEY5uEzx8DGMAoLkvQXtrKs+SgAHr0v+gqag733xHBg6BLDz+sacACwbN0ourzg+XXR3zJyKaAM9MN0BB2Ud3VvjC7JiacN/RukNyJAplpqeKm0F9V2rV9pZItv/epM65RUxolh/VG1gmXbLqQ28H6ckBjdlJsLGL9+CnlXLHW62DRGqU3rVCm9kED7UAjwgyHLtnuM8qqWGgpr6nreO8dYSLIY77QfKJYWLbAlRMbD2REc0xj2kjNYkppjibMZpCoAlMLati3XrvgsYm8LgWBtldxovhoddFR4jzZwTP2wWk/RiaB1UoU6kMdyUfvOv6jZmj01ado5dJ/1pUunceBG3//8V2XkwZo8Cmoq+O8bJ2MvKAzY8IaseaHj3cXkLnyA/dctbzJt7qYHAMieUUi0PIIvlfbik+u7O5fta2L5e6/3I6jIP/x19g4ljuCYxnho4nSOPVrN9iteJpgAmnBibduWqVs/48bEUiByLkX+cnRFeqP5Gu1BR+EI5OnyB21cj5nLdK8Yu2hGPUi6xyTw//1tOTPufCDgsUNCZnRtZWfJfvos3cqnAnoInRfWXK1t++kL0wxvXvj0nlzueqk9qzLy6sYAWBCvveKc0F5xzmzIYnX35RhtvIJB4m1+OXsHi8rtzZBl2r2b2npLk+cet+RDhifupbVLuruKhnP6wU5A5MaIcKBye3PtCs3gxqmGBjeSWv/ws9f54rbuXrdPSF7ilq8OHIFJiXn52KMg6MjIQB677qKI47seOb87w4iVkxs9hqdAHquykBXbHIlRAXvyhbQj7fBP2jN9SNMNn462lOXXLmPyknsMNbyyfRdHn87lyvQcp/uK2qLVAhxBCev6LHG2T4/Iv0VL82JboMgQjfays0x+8hEe+fUaJiRFX+jpxVG5dHxqn0szTdPGXhsMxT3dkfJWxEVosJt7xm/k/avrfK87JR3RmxQ813AjqXVkYg0jExsbbMVz/leJheZfFGEz0BUTIGN9KQBXHnIPnnAE8lyZnsO5GyoA2DN0paHaHNj2FjbpRtmuth/ZD04CGups9+wBjj4dWOd6yIxu8ayrGPezPL/awoYn2PnL9a9w+8szyP7VbuwGFY5mG7eT4rIsV/dj7z0xWOJsbL/iZWet4Zpvx8Ir2ghUCRu/MEQbgFRXk7zqcwoe6wC60b07V+v0eWvWcNK9dAAZRXl6rFttpdtfpxJ/vGFRqmpf2+gbzy+6buaZeRMahE+fuWew83fKsvCEVj+asj9q223lfDnjXpvFgntfbxDh5ytaYMosupw3fvhMhz9wfW8JRyBPCpB8QPOjzWaS4Ya3eNZVlOdcpO0W7Q/WWxlTW3bSOdazzlUZeVyZnuNmR3wlJEb35IyrmHT3Ro+FuNR2gdxND2CvtgKa8Xgqtc4RfmBcLJ/fOp8h52fR8+XDhs4c4epAXdejrrWLjci/BV5JJWG9cca2MRx5lnh3NX+uGuXmqF02aTDDL4/cK3qPNTVYNn1B5ZiBnOrne5Ga2KKE1J8u4aGaX7it//EN3zoDF5pyQfMVuVDJTe8/zJujvUdGupbVuUPeZWKLktCc3E/sFy7Q6dmt/DJpCr8et9ajjqaeqy8udKdTvcCUHzKWxEQOPdQPexzcM06zVY5AnsbKmGPmiJbdBsPQEGkJzWFMTExMTHwhJDXd26Z+7LGWW1BTwfivpmheCvpgJiuWXsVTN7qHfDqcuEf8bTIWA2q6F0flUp4e26BdqdxexbiC26iqjUW92NbQJgVfeTRlP9VTP2bzwjp3toS7jnkcI8BxPTEllWGt8Ry5NoGkzMG0uusI+V4CTu4qGs6R8lb8outmt5rbyMSaBq5FAN30mm6osJ8/T88HtvHTJdNYfu0yhie450j9svr6h0OZGGS8frBkzP6MOdYJvD6gYbh66YWEJp+raMbI4AVrairHJvRk47TfO0e/LQUatAAAGnJJREFUayo4xoFjJpyS3NA9QUEZXceo8WmxHzbYVlBTwbS9d5C2IJ6avpnO9S3bVARzyoBxaJUY5dHx/aStgtnFI7GMrySu5ChGdZoFQlrsOeTqHzfppH241o7ljhpsJ0Iz4r2D+o72nowmaHm6uPQKAE4/2Im47bt4Zt4EDtzyCQBZ8cc8dhR+eCGWZsfD47KVNWU7k5fcw70DP3Vb/3+HfkT7MflR48LoIPNXntsb20PUaY1GHDOYfOUSpLW2PJk/vzqK9HpjKbjaCAcFd2jl0NNMOIEGnARldC1tUliwepHHwWwe3D+B2j+0Z9Oapgcwtomd/bWVqDA4cHsb4d6VcnsVs4tHcmhQBRCZPwVPHKxsQ7m9qkGQxtTkYob/+WX++8bJUFJGfExgnS2BYqkVp9N5Y8w/4chTcLhaZcz+jM2ztevZcNcd9Jk7v8F+Dy+dRcaz4esszJqyvcEQjSnUjZNrbZNieJ5+X7C0aIFKTHBf6fCRFcHaKhni4sIeyNPYDCbP/mEiHd7aB+3S3HdqxEbUJ5iAk7CNvfBW1tucWliLL7MH7K+t5L9vnIylYFfIHbi9jXDvyriC27CMrySaDC7A0dFxjFt3m8cgDYeTtk0UnWMsYOA4r63WfcXMj3wYirG2lsby1NtxupzfGdEOoI4fVLOg4wcYmaffF/b87jLev2mB+7qaNGfgw9EV6bzZbyltrI4nPTy+5Y3NYPLe47/n/KMNu7MasxH1CSbgJGxGN8kST5IP3XSLz6bz1pSRmsENIp7ZG55GuAfNzanHGq02E1NSia3kaMjPHSy2kjNYHuxF37l3eHXSjgRSXY3NZXaASB8n1HRNKLmkQsCjCYm3NXjeMmNL2ftJIdX2WCYkL2lQbp02oCR0Hjh2vbm2/gwm3tb5wuKz6ay7XxvfJHFH4AEnYR9P1xOOgIOyVZ2IP2snccu2sIUo1h+pPmfxdJIOCr12lDn9CaPZrca+6zvOleQ2ndAkLEQiOOZSpueyWrKT3AMK4lSsS0d7w4rCyZqWqC07oypMOfPd+2iz3b3WGH/WTuKmbUBwNiMoo+tw4rbH+bdfqj7TQsr68M8p1nZHGTmLpzuXM5ceovbwkag2tPXp/J6FnMPaNdhzyn1yJo+kc/z3iX1F7QDIiuCwnpcSjQUUeCKUMzK40nGTNmNFjpreRErPZIdx0PegjK7DiTuasX+dTxeXyBgjpwMKFQnrv6CL7r0kV/ejW/mUxnfA86wNJt9fPM7a4cPsIuHAEVDQtabpspqU36yBF0EoNXTJC2z/cD43ZnCEiYmJiYFEpE3XJHDUlp1kbYm0iu83S78Ywj+69ALwyYE+GvBl1g6jMcuqZ0yja2JSD9dR79LNzjOTEGM2L5iYmJgYiJIQTY9hYmJiYtI0Zk3XxMTExEBMo2tiYmJiIKbRNTExMTEQ0+iamJiYGIhpdE1MTEwMJGqMrlLcrhT5SlGhFPuVCtWMRKFFKVKU4l1d50GluDPSmjyhFHFKsVTXeF4pvlKKGyOtyytKrUKpYyh1DqUKUKrpWOcIoBSrlOKYUpxTigKliEqdrihFT6WoUopVkdbiDaUor/exKcUfI63LG0HZKxEJ6QckJoB9RoAcBPkxiAWkI0jHUGsLkdY1IH8BSQIZAnIW5PJo0wrSHGQOSIaepzeDnAfJCLdWAb/zVeBygTj9dy+B4wJXRFOe6vtcDhKn/+4FchwkrDoD1eqy74cgm0FWhf3eB6lV3785SDnINdGoNVh75c+JikCeANkNUgqyHCQeZDjIEZDH9QL4hp7+ZpCdIGUgW0F+1Mixt4LcG8KMDItWvTBcBMlyWfcGyO+iTauXc30DclvAeQtFAk8I7BYoFVguEC8wXOCIwOO6sXxDT3+zwE6BMoGtAr5phWyBYwITojlPQbJBjoEEpNMIrSC3g6zV/4CDMroG5uvPQQpBVDRqDdZe+XsRu0A6g6SAbAGZq19ELchzIHEgCSADQE6CDAKx6plYRF0NYRHIIv23VTdkvwLZp2fIQpCEIDM8HFr7g1TWO9cskPeiTauH87QDqQLpFfCDpxndXQKdBVIEtgjM1Y1urcBzAnECCQIDBE4KDBKwCvxc399Rm10ksKje8RcJXBAQgR0CSdGYp/q6C5pM2QESkM5wawVpCVKgHztURteIsvoxyJxo1EoI7JW/F3G/y/JokP36RVwEiXfZ9grIb+rtvwdkmIfjpuuFdztIB5C2egbNCzLDw6F1KMjxeut+AZIXbVrrpYkF+QfIa8EUZN1o3u+yPFpgv250LwrEu2x7ReA39fbfI9CoVt1ADxF4UiA2ivPUita89CRIQDrDrRXkDyCP679DZXTDna9dQGwg3aJRayjslb8daYddfh8E0vXfp0SoctnWFZipFGWOD9DZJb0rlfr3H0U4JsJpYD4w2k9tRmgtB1rWW9cSOB+FWgFQCgvwBnARmBGkzka1ItJAK0qVOT9NaAVAxIbIp0AnYFo4dAabp5pMbCKEQmdYtCpFP+D/AQvqb4s2rfWYBHwqwoEo1Rq0vfJ3lLHOLr+7AMX6b6mX7jAwT4R5TR1QhFKlOOLhGMEScq1AARCjFD1F2Kuv6wv8Jyil4dGKUihgKdAOGC1CKKa49UsrIj5p9UAM0D3AfSFMeeqBYHVCeLQOBzKAQ0qbsioJsCrFZSIMiDKtrkwCfhegtvpEp73ys7r+Lfz/7Z15dFRVmsB/VZXKHkJCgkmAQAIkxAFFDKHDYqMeQA6LaAMNohyCTBRIu4DN6MHj1qZbR9AZWhYjmwiiwCgtLkN0mjAISBpDVIZIAkkIEJSwBLKnUvXmj1dVqS171XuV9v7OyYF6daveV/e+9717v/stUl9kG8khkP5snq5fcGibDNJ5ZBuJBnkDagpIIS189ysg/QOk3iCFmb/7T+2VTWFZP0T2YAgCaQxd9F7wsKwbQPqWLtgc7f5k88KPEvQ123QPSfBnybKRZt82WYLzZpuuRoIgCaZI4Cwr9JZgjgTBZvPCJAlqJLjfm/rUfH3OQfZc0YE0CaQakDolp4dlDQQpyuZvFUh7QIr0NlltPjPa3J8ttvEGWbuqrzr6Iyy7gZUgvWceWKcfYW5/n1mwSuQd3t2WH2FWBhts2uqRjdWVyDuKa7CxuXSywz0lazhIe80XRxlID7nh4nC7rCD1B0lC3jyrtvmb12l57b0XKiV4T4JAl0pXbn+fBP8wt70kwW6r0oUNEmww/z9SgoPmdjfNiv1fvbBPI0E6aG5303xTd1pOT1+rDp97Cfd6L7hdVpDewexN0NU/b9ZX7U7tqNFQCiySJL5u1wdURMjqITSaUmARkuTVsnanPhWyegZvltVrItIEAoHg14BQugKBQKAgonKEQCAQKIiY6QoEAoGCCKUrEAgECtJqcMQE7Syvsj18Zdqtaek9IWvnaUnW7iInCFm7wj+DrN1FThAzXYFAIFAUxZRu4cZkGrIHUDk/ValTCgQCgdfR0dwL7UYaM5yihc1f/9E960nx05MUv4SenjppF9ANjqfg2XC7Y/32aQnYm6uSRAKBQCkc7//Bm5vQHM73yLk8pnSvJwRQMnm9zRG9p07VZbS3J1GwOISSyVl2x+/q+wB1pAB4nfLVBgZS9uRwTH6ttGmA2P/Mx1Rbq5xg3YS6GSlUDG/78o/MbwK8b/wF7sUQE0rJ5I3W14nB8+mnH4EuJ8/t5xI2XYFAIFAQj810XbH80gh6FHvPJqP29iQATqf3oGT6O07v/++wT4ibJtcdTNirqGitoouM5NLswexf/O/09QlusV1BYy3LN88ABWa6jZOSqY5pezUTkVeJ6fsCj8vTGo2TkvH7wyUKkva12TbuS+8bf0cc+z643EBA3jkqpg4CIPKzMxgrKtQSr1tyetw2kk4tITbH/d/tVqWrGxyPISYUgOr+9h4TWTdiyM0cSc+Pj7rzlF2idEYYAMUPrGuxTY9eNQBohw7BdPInReRyhU+/vgA0xkdSkeTPiZXrkFOktoy/xkRNygACc+owVXU1z3rrlM6Cj+5dQ4pf881/2VjDezdu44mwn/DTyMfjPk0naXU8AMaiYo/K1BLS8it81Q6FC94z/i2hSR5KnxfPsH1AjvXYw6XjObUtibwXZPPehOI0tAfdo3Rtr0O743lnPH6NeQpdpHxPKYVblW7Bs+F2dhFbdj8+icCDx9x5ui6hDQnBGOB61l1iqCZS50Ow1p/vU3YCMP7NGfhNVFJCe4oXxgJQ8FjLDwhH4vTBHHwniwlz09AePOEp0QBIWHScOWsz+HxKc6GCN3+ZyMWpgQw+8guTA6/jp9FTMj2LOL159vioOkq3I3jL+Duh0aCLiODurUf5Y/hZu7e2D8iBF3I8ctqWrsPfPpZOUG6pfeOGBoyVNzwihzu5MmWQ9QGlBIqaF7yJ06/dyrfTV5lfBdm9tyBjGQFPX+S/h3yuvGDdmMRnT7H8lRnNB5qaMF67QtboVIoOFDspB0Hn0UVEkH7kKJMDr+MNm9Rb336Tesl+i2hO/qNEzfB+pas0blO6hRuT+eie9TheACWGahZkLCMwrwCTu07WRQo3JvPRvW/TW2evbK2y5hRQ94dedu9lJe5g3mdphE8tVFLUdpGUtYS+B+o4M1fu+5LpWW18wjOYqqrAxRLTWFFBg0l9xWAhYJk/t7861zqLfbh0PL88F2d9f9aG/aSHlrf0ce9AqyFRfxk/TaD10O25cwFozA+jIL39K6KOEL+5DIAJOWlcTfK3zhDj9M6mrt13bGTXD3e2+n2bcseSsOi4+wVtB5czRgPw3BM7nN5LylpC/OYymjxwXrcp3f6xV+zseQC7qkN585XH6Zl9AlNDg7tO1WVcyQpQL2kJyi3FWFWFZnUiE1ZMs9r+EvRB3B/7A4dQzvbTGheaqpn2+gp8aiXiD5bTVFyK//jRaovVLTCd/Ilb3hjOyAS5nmRwuQHfg8fR9Qzl4tYYxgcW4bj66Q6sSMoGIK9ff6D5Gok+VYTRTedoOn8BAO35C0SdiuQO3RL2/ZvrDd0EfRDPR7RuB/86doibJOs49eZ51exg59l48DnJ+lvdjVuUbmlmKi/032V37I1rA/lgwyR6bz/i9oqTSuC7/ziFs5MhqfnYxJAf2bw2g8RnTym/aWCzL5ldq+fpTSuI3SL74DYB1xam8pvJPyorUzvQhoRw+rVb+VPI23jDMtiC5nA+4Yfl/0tjhlO4KRmtn5Hjd64lzGYFlFY2DoC67dH4UaqCpM7oBsdz+sVQonT2x+eFXLX7t8qkJXqP5zwXjBUVRG2pYVKPFS79xU1J1Zwet80j5+4qrd0vcZ+mMySv0mMrc7co3UemHbAO9MsVtwKw5/3xxLx9xKmtK6d0b3Diz20wMOeLp0msPdVimxQ/PZ9PeUu2WyqsdC1O+nFfLsL3Zz0D/nLEelFUzk9laPpJtsQesvvMdWMtyQeXklh+w20znY6iCQzg8ylvkeQb2HZjhTGOl4vinl/cRMk4ywawvZw5/5cIQMI27/C6sQby3JOFo6xqYKqtpe9fnO9zkB9mcdWLrK8tUanewJUxBqf7xZbSB8LoEyaCIwQCgaDb43bvha3HZbtiwir7p1/tg6OoD9XS8+ELTk7pF5qqmXZzBdG7ijy2FNL4+VE56w6Meg139bJ3Xcup05J2YDEJS4/ZLSmCC3x5Y9RAANV33i1hqI5O+rUPjmLsk8dYHW3/RC401DDrxCISM4pVc9vRRUZyaeYgQrTNvfrGtYEEF/iqIo8t0pjhnF8srx5aWwIPGvALANfSUvG/YSLwY3XdHq+M6EnJdHv3puWXRjAi+Jx1tQny+M/JX0SfBvU2BDWH80k43Pw6+4dhpPip7+vcOCnZOq6usGxEJybNp0ecnKAruNyA7373bPh1SelqfHwwjRpKb312m20eydzX4o5wX59gTqxcxwjjYqL+C48oXm3PULa8+qbLZe5LZ6e73EGNWXWEzUGTAPhjB/xjlaC1fi001DDvxzSiZhQoblZoK4hj8+5JxK5yvRxVkqKFPjYmhZaxBlFkwqtXhnDoY3U2Ui396iro6Ksdv+HruxKZZ/bGANh1405Vxr+95DYYADhXFkGCwrby9gbHnB63DWSTvuzhUn8HmiYJ7bGTSE2d92voktLV9grnrR3rWrTXaXx80CbEs+aDtSTomzcnrhtl222NZLLb9cx7YT0p9YsJ26pMyKJFjuu1AUQpckb30Fa/zjqxiKgZyoba6nqGgp8fRUv7AVA439nZ/EJTNdpGRcXq9jj3q/zwN0omzjbVsWfRRMJfLCdnaPMSqNpUz7m6XkCNGiK3i9//XfYcUctdrDVKDNXUS1qidBCmk3Xb9gE5sDOHQkMNT01Ow1RY3GnF69HgCNOooaz5YC0DfQLsjicfXApAj6MB5pmQOljkSMwo9toZgSva6lc1fs/FrTF8OHwTvXQWXxVnl6tpr5s9LpQVrVvTUr+ebarjqclpaAtPAn3tPjOz8HdoZ9XhzUrXm1mQsYyg3FJOr+7D2Xu22L030CeA//hyCxkPLe106kePKd3aB0fxSOY+u5kYWGLvrwBwZYy6Pq+mBtnnpiWbZ2lmKn/9/btKitQmLfUrtP17PElYYF2bHgr6Gslr0kwmvXaNOEM6INvwsm7EsPvxSXZtLj9dbw2gUINrnyWwe9hGl2Pd38eXu3d9R4NJz+zQHdg+5NYM3MWuA81BCd8sTEY6flIJkV2i6xVOny8aSA87jLf6P9uOf2BeAcaqKhJfDiauuvkaAdBptCTog5i5MZv3V07rlI2/S0rXVHmDtOeXseyFnVYH41fHfgJA3vD+TrbG+E8eQyNB7Vp5rjOul7PAUouVhdxL4qH5DN7c+vKgMcrAxECDMgK1gaXixtgnj7m0jbfn93gSzeoIRsYstjtW3V9jFxml1Ni2RfFrqYSfgiHrKwEYeWyxvEnmkBvk5vxku9ezQ79j995F9FlQrsiD7f7YH1wqXAA/jd5mc9e+jWNQwoSgVHXdlHx8WHbLZ9YIULWvVUeWXxpBbuZI6/hbVmLGomKGrJcdkOONj1H8QHMmwvTQct4N1XbKaa9LSldqaCB0+7cUrogGs9J1dNC2ZeCQcsZGnOXFSNe+sJ52SrZFWxCM5nDLGzrlz4xmQXKOApK0Tvkzo6lOamT8v8gzFUcvBZAv4n7rfaiPlIfz/KZk2Zd3pXK+pb77j2Nbd0MaM5zG+70nCtFCaWYqLz2wi6/H32r1wQ0u0BC+pe0ZS4I+iA+Hb2K534w223YFdwWUXDbWMHbHMwwuPu+RcNbO0ta9pzTfXY1tccZqSUOamJVEnC7dLSH2bjEvvL/vbuJmVrhUtLa42jHMbTBYjepJq6+olu7PlssZo5m/YL9qbmK2VSEWzmyfHBfH+xOZegmAkmGfyDvtK9Uz31xPCOD7lK3W17fnzuWWwjrV5LFgCeSZF3KIlwPk63VPwXindpXzU60POiXx6deXoqX9+Hb6KqfcIK2RVjbO+hCxoKnXkZh5kqZumnLRE1zOGM1DMfvd8l0376shtKTjARQiOEIgEAgUxC0z3QErj/LuyHHMG9rx9PrZVcNIeFR2G1F0x90hIYQuMtKaaf+5J3a4TIKhFJqQYPY8tqrdobO2/oTeSu+3/NEc9mxO39awBMck+DdvjH1zRQ588b8qUfvgKII+z28OoMlwHXDi6YCDxvhIs7td+2e5yy+N4NS6oS5Dlb3SU0TFZCy/S/+708rxzl5l5D44ys7E4FiN42qyyam6zNRBJ/kmdlSHC+26zXvhXFkEuYMN7YqttjhGZ1cN429ltxGO8ukS66ObMP32DuvrCps0da4oNNSw68adsv9jFxyj20IbEkJNygD8NZ2/XQoNNar1q6MTf4NkYM31IehqDKomPnIVHGMb+JB1I4YdhqnszFzlMk0heG/AwaG/jiLMS3JDeDt/K7uN2aHf2W1Qro7OIyvzZ3ZfbfZecazG4UhXKuG4Tek6Vg4I0Zpcpnu7bKxhzhfLABi89JgqigHMLiDT298+/fQ8/CaWIvs+es7/sWnEIA6+k0VbpXhsudBUTZWp2VI0J1/54AgL1soCZq+FYoOBA/cORPpFPZclAEwSpw29iddft5YOsiU9tJz0VvpdrYADy9ha/HRd5YDWNnXHPH7qED61kPTseXbBJGAe/51bWvhUM7ZBKYGHOxcS7lY/XdvKAZdmDnIZ+DB2xzMkZso3oFcufboh015fQfSeM9bXfRrKvW42pjbGK12rYKFWwIFlbM88LZtCHCP9FmQso2f2iW6ZPrU7YhuU0tk+d6vSta0cEL1LYsLJNKc2g4vPd5vd1LhP0xm0UzaFBFyt86qHhKXKhe9Ng5ykWlR7bR1JwlhRwf8sSOX7Nf1aXTraYqnGEPO8hPHqRQ8KKOOTd4YJc5vvm+hTRZx9ahBvz7TPE2Fb5cSbCgR0Bxwrh7SGZfx7vyV7AmmaJFnhqpV7oTWMFRUuK5Cq5S9oCeQw6tvvoT/Eply42gp3QsE0ACq3yzZTnUGSZzgNDV4zqy1/ZjQLZzW741grh1Sqt4HmiHT8JBdfTmbCipB2JT25eVVezkedVCZHgKmqyq6IaNFrsl+xbZCOt1ZkaS8LZ+1nW41sP41RIfmRY+WQ1rC4OdpuAnd1VfGrKUxpCeToCGooWt/iCpKyljgdtyQxD9/bbLj3tiVldVKj3dK9sD6a0O3fep2cvvuPUxeQQtJw5352pF++umEFfUZcsvN/7+4VWUBOk7ouSd3MR7aVQ5TmV6N0uwtN5y8Q+5JnajN5mojDeuJorhTg+7OeAXjnrnrA3lxiO+7hqDhXs2OIO9/cp8EFvi4rsnQn0srGEXHYOypIqIEIjhAIBAIFETNdgdsI33yU8M1qS/HPhRo2T7fT0MCc/EcJC5Tto3Xbown/FfsVC6UrEAg8irHyBlEzmiM8vaWqsloI84JAIBAoiEaSuuseqEAgEHQ/xExXIBAIFEQoXYFAIFAQoXQFAoFAQYTSFQgEAgURSlcgEAgURChdgUAgUJD/B7UCzbm601T5AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 32 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds_test = create_dataset(test_data_path).create_dict_iterator()\n",
    "data = ds_test.get_next()\n",
    "images = data[\"image\"]\n",
    "labels = data[\"label\"]\n",
    "\n",
    "output = model.predict(Tensor(data['image']))\n",
    "prb = output.asnumpy()\n",
    "pred = np.argmax(output.asnumpy(), axis=1)\n",
    "err_num = []\n",
    "index = 1\n",
    "for i in range(len(labels)):\n",
    "    plt.subplot(4, 8, i+1)\n",
    "    color = 'blue' if pred[i] == labels[i] else 'red'\n",
    "    plt.title(\"pre:{}\".format(pred[i]), color=color)\n",
    "    plt.imshow(np.squeeze(images[i]))\n",
    "    plt.axis(\"off\")\n",
    "    if color == 'red':\n",
    "        index = 0\n",
    "        print(\"Row {}, column {} is incorrectly identified as {}, the correct value should be {}\".format(int(i/8)+1, i%8+1, pred[i], labels[i]), '\\n')\n",
    "if index:\n",
    "    print(\"All the figures in this group are predicted correctly!\")\n",
    "print(pred, \"<--Predicted figures\") \n",
    "print(labels, \"<--The right number\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建一个概率分析的饼图函数，本例展示了当前`batch`中的前两张图片的分析饼图。\n",
    "\n",
    "备注：`prb`为上一段代码中，存储这组数对应的数字概率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-09-04T06:46:59.943697Z",
     "start_time": "2020-09-04T06:46:59.803412Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Figure 1 probability of corresponding numbers [0-9]:\n",
      " [10.731268   -8.178983    0.7688376  -1.5208994  -3.4331348  -0.6836271\n",
      "  3.7032425  -2.7914028   0.43636245 -0.42814386]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD3CAYAAAC+eIeLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXiU1f3//+d7soc9ELaEJIDsSljihksAURHxV4tarNhS69L+qrZq7aZVUOtSW22tVq1FqfWDW9FWrVqxVca6VB2UIKsLO4oGEEggZJvz/ePcaIRAMuuZ5f24rrlIZu7lnSG8OHPuc58jxhiUUkrFh891AUoplU40dJVSKo40dJVSKo40dJVSKo40dJVSKo40dJVSKo40dFVCEZGFInJBjI5dIiK1IpLhfd9LRF4RkRoRuU1ErhKROTE47wwRWRDt46rklOm6ABVdIrIWuMAY82/XtSQaY8x6oGOLpy4CtgCdTZQGrItIGbAGyDLGNHnnnQfMi8bxVfLTlq5KZ6XA8mgFrlLtoaGbwkTkOyLymoj8TkS2i8hqERnnPb9BRD4TkZkttj9VRN4VkZ3e67P3Od63RWSdiGwVkWtEZK2ITPJe84nIz0XkI+/1x0Wk4CC1fU1EFnvn+khEJreyzUAReck73hYRmSciXVu8/jMR2eR1D6wSkRO8548QkYB37E9F5Hbv+TIRMSKSKSJ/AWYCP/W6HCaJyGwR+b8Wxz9WRF733rsNIvKddrxPr3h/bveOe7T3fr/a4rjjRORtEdnh/TmuxWsLReQG7++tRkQWiEiPA/8tq2SjoZv6jgSWAN2Bh4FHgcOBQ4BzgbtEZO9H7l3At4GuwKnA/y8ipwOIyHDgbmAG0AfoAhS1OM8PgdOBSqAv8Dnwx9YKEpEjgL8CP/HOdTywtrVNgZu94w0D+gGzvWMMAS4BDjfGdAJObnGMO4A7jDGdgYHA4/se2BjzHexH/luNMR337Y4RkRLgeeBOoBAYBSxu633yfhaArt5x39jnuAXAs8AfsH8ntwPPikj3FpudA5wH9ASygStbeW9UktLQTX1rjDFzjTHNwGPY4LreGFNvjFkANGADGGPMQmPMe8aYoDFmCfAINkQBzgSeMca8aoxpAK4FWn4s/x5wtTFmozGmHhuOZ4pIa9cNzgceMMa86J1rkzFm5b4bGWM+9LapN8ZUYwNqbz3NQA4wXESyjDFrjTEfea81AoeISA9jTK0x5n9hvG8zgH8bYx4xxjQaY7YaYxa3431qy6nAB8aYh4wxTcaYR4CVwGkttplrjHnfGFOH/Q9jVBj1qwSloZv6Pm3xdR2AMWbf5zoCiMiRIvKyiFSLyA7g+8Dej7Z9gQ17dzLG7Aa2tjhOKfB376P4dmAFNhh7tVJTP+CjVp7/ChHpKSKPel0IO4H/21uPMeZD4DJsuH/mbdfX2/V8YDCw0vv4PrWtc4VSYxvvU1v6Auv2eW4dX/3UsLnF17v56sU/leQ0dFVLDwNPA/2MMV2Ae7Ef8QE+AYr3bigiediPx3ttAE4xxnRt8cg1xmxq5TwbsB/723IztjU90usqOLdFPRhjHjbGHIsNfAP82nv+A2PMN7Efz38NzBeRDu04X3trPNj71NZFuY+9elsqAVp7n1QK0tBVLXUCthlj9nj9rue0eG0+cJp3ESgbuI4WAYgNnhtFpBRARApF5GsHOM/9wHkicoJ3Aa5IRIYeoJ5a7EWpImwfMN7xh4jIRBHJAfZgW+zN3mvnikihMSYIbPd2aQ7pnbD9vZNE5BvehbfuIrL3Y/7B3qdqIAgMOMBxnwMGi8g53nGnA8OBf4ZYn0pSGrqqpR8A14tIDbbP9osLUMaYZcCl2AtxnwA1wGdAvbfJHdjW3wJv//9hL+LtxxjzFvZC0e+AHYCf/Vt/YIN9jLfNs8CTLV7LAW7BjrPdjG3VXuW9NhlYJiK1Xl1nG2P2tPdN8GpcD0wBfgxsw15EK/dePtj7tBu4EXjN62o5ap/jbgWmesfdCvwUmGqM2RJKfSp5iQ5RVOHwRjxsBwYZY9a4rkepZKEtXdVuInKaiOR7/aO/Bd6j9aFeSqkD0NBVofga9kLQx8Ag7Mf2hP+oJCKXi8gyEVkqIo+ISK7rmlT60u4FldK8C3CvAsONMXUi8jjwnDHmL24rU+lKW7oqHWQCed6NGvnYlrpSTmjoqpTmjRP+LbAeO+pih3cnnlJOaOiqlCYi3bB90f2xd4N1EJFz3Val0pmGrkp1k7DzT1QbYxqxY33HtbGPUjGjoatS3XrgKG+omwAnYOeFUMoJDV2V0owxb2JvYX4HO67YB9zntCiV1nTImFIqIS1atKhnZmbmHOBQErOBGASWNjU1XTB27NjP2ruTrpGmlEpImZmZc3r37j2ssLDwc5/Pl3Ctw2AwKNXV1cM3b948B/j/2rtfIv7voZRSAIcWFhbuTMTABfD5fKawsHAHtiXe/v1iVI9SSkXKl6iBu5dXX0g5qqGrlFJxpH26KjGI7F1EsggoALp5j71fd8XOoZsBZGTQ1Bgkwwc0eY967Ly3W1s8NmNXZPgYqDamzVUdVCITGRvV4xmzqK1N5s+f3/nKK68sCQaDnHvuuVtuuummzW3t0xYNXRU/Ij7sqr5jsGuYDcAG7UDav8YYAEF89dgQbq9GEdYAy73HisB9y6vGDt69ksqKxlDOrdJDU1MTl19+eckLL7zw/oABAxrLy8uHnXHGGdvHjh0b0oT4+9LQVbEjcgh2uffDgQps2Ia6Vlm0ZGGDfjBwOpjgsJK6esCHP7AUeBdYBLxMZcUqRzWqBLJw4cIOpaWl9cOHD28AmDZt2rb58+d3HTt2bEStXQ1dFT0i3YETgZOBk7BzHSSk7CyzLj/X9Pe+Hes9LgDAH1gHLABeAP5DZcX2Vg+iUtqGDRuyi4qKGvZ+X1xc3PDmm29GvDKzhq6KjF0DbCo2aMeQJBdn+xU2bMZOgtOaUuBC79GMP/AWNoAXAG9RWRHqIpcqCbV245iIRHxdQENXhU7kUOwKuGdz4OBKaBVDdte3vRVgL9wd7T1mA1vwBx4B5lJZ8W6MylMJoKSkpGHTpk3Ze7/fuHFjdt++fSPu/9fQVe1jl1afAXyTEAeDJ6IJo3fmhblrD+yqyJfiD1QBc4F5VFboar4pprKyctfatWtzV65cmV1WVtb45JNPFsybN291pMfV0FUHZmflOhEbMlNIkq6D9qgcVds7CocpB34P3Io/8E9sAD+v3Q8x0o4hXtGUlZXFbbfdtn7y5MmDm5ubOeecc7ZUVFRENHIBdMIb1Rq7vPpM4BJgqONqWiUE60FCGTLWgtnV/PKifJ8g0a0KsGOD7wb+QGXFjhgcP21UVVWtLS8vT/hPEFVVVT3Ky8vL2rt9yrRcVBSI9EHkduwNBXeRoIEbqc75wbUxClyA3sD1wDr8gRvwBwpidB6VpDR0FYj0ROQ24CPgcqCz44pialDxnm1xOE0X4JfY8P01/kBhHM6pkoCGbjoT6Y7ILcBq4Aog3ItLSWXcobXBOJ6uI/BTYC3+wO34A33ieG6VgDR005FIPiLXAWuAn+HuLjEnJo6u6eLgtPnYTxGr8QduxR+IeJC9Sk4auulG5BxgFXAt0MlxNU4cc1hticPT5wI/AVbgD5zhsA7liIZuuhAZgcgrwDyg2HU5rvh85tPCrk2JcHGrGJiPP/A8/sBA18Wo+NFxuqlOJB+Yhf1om+W4Gud6dm3cCPRyXUcLk4Gl+AO/Bm6msqK9d8qlHbkuulM7mlkHH/f74YcfZs2YMaN/dXV1ls/nY+bMmdXXXHNNu9dCOxBt6aYykSOBxdgLOWkfuAAjB9bVuq6hFbnY/xiX4g+c7LoYZXk3R2xcvXr1srfffnvF/fff33PRokW5kR5XQzcViWQhcgPwGjDIdTmJ5PiRNRmuaziIQ4B/4Q/chz8Q8T9uFZnS0tLGY489djdAt27dggMHDqxbv359dlv7tUVDN9WIDAPewI4RTeSAcWLC6JpkGC97IfAW/kBK3pySjFatWpW9fPny/MrKyog/KWnophKRS4F3sHPDqv2YpjGDd5e5rqKdDgMC+APfdl1IutuxY4dv2rRpA2+55ZYNBQUFEY/x1tBNBXbc7cPAH7D9g6oVOVlmXW62CXO+Bic6AA/iD/wFfyCtxlInivr6ejn11FMHnnXWWdtmzpwZlcnsNXSTnUgZ8Dp2ykV1ECW9GiK+8uzITOBt/IGkn1IzmQSDQc4+++zSwYMH75k9e/an0TquDhlLZiKTgEeB7q5LSQZHDN2VzMOxhmH7eS+ksmKe62JcaGuIV7S9+OKLHf/xj390HzRoUN3QoUOHA1x33XWbpk+fHtHscRq6yUrkSuAW9GJZu00YXZPvuoYI5QEP4Q/0pbLiN66LSXUnn3xyrYnBHL7avZBsRHyI3AX8Bg3ckBxXXpOwC2WGQLCTpt+OPxCr6SlVDGnoJhORbOBh4GLXpSQfU3NIUX2R6yqi6HLgYfyBiMeNqvjS0E0WdjWHfwLTXZeSjLp0aI7lxOWunA08hz+QlhMXJSsN3WQg0h34D3a9MhWGIf32RGW4TwI6AfDjD0RjzTcVBxq6iU6kEHgFOMJ1KcnsmMNqU3kxwNHA6/gDZa4LUW3T0E1kIt2AF4HhrktJdo4mLo+n/sAC/IGe0TiYiHQVkfkislJEVojI0dE4rtIhY4lLpBPwPHaZbxWhow+tLXVdQxwMAl7AHxgfhZWI7wD+ZYw5U+wFXPfD7fyB6N7eXlnR5nCwLVu2ZJx77rmlq1atyhMR7rvvvrWTJk3aFclptaWbiETygGeAI12XkgoyfOaT7p2bu7quI05GAU9HMkuZiHQGjgfuBzDGNBhjUrVP/KAuuuiifieddNLONWvWLFu+fPnyUaNG7Yn0mBq6ica2Kv4OVLouJVX06ta4yXUNcXY88Dj+QLifZAcA1cBcEXlXROaISNrN/bBt2zbfm2++2emyyy7bApCbm2t69OjRHOlxNXQTzwOATmQdReWH1EX0cTBJnQbcH+YNFJnAGOAeY8xoYBfw82gWlwxWrlyZU1BQ0HTWWWeVDRs2bPj06dNLd+7cGXFmaugmEpGrgBmuy0g140fVpOu1i28Dt4ex30ZgozHmTe/7+dgQTitNTU2yYsWK/Isvvrh6xYoVy/Pz84PXXHNNxEPzNHQThcjXgV+5LiMVjR+VFBOXx8pl+ANXhLKDMWYzsEFEhnhPnQAsj3plCa6srKyhV69eDRMnTtwFMH369M+rqqoivqCooZsIREYBD0HK3TEVkQ3ABOz0WiOwl9P3twP7Sbrc22qu9/wq7FzuI83u+jf6AzQ1NTHpih+we0/E10KSza34AxNC3OdSYJ6ILMFenLsp+mUltpKSkqbevXs3VFVV5QAsWLCg85AhQyL+5UnXj12JQ6Q38DR2wmrVQiZwG/ZzbQ02Qk9k30HLf/SeeQZ77WcItofmT8At5GT1WXfnkz8sGz/qVu55+gm+ddIU8nPTbp73DOAx/IExVFZsbM8OxpjFQEVsywpRO4Z4Rdudd965fsaMGQMaGhqkpKSk/pFHHlkb6TE1dF0SyQAeB/q5LiUR9fEeAJ2wLd5N7Bu6go1kA9QCBdhf6yygjl4F1duzMjLZXlPDM6//lxd+c2ecqk84hcAT+APH6zLv7Tdu3Li6pUuXrojmMbV7wa1rgONcF5EM1gLv0trA5UuAFUBf7LJid2B/rS8GbmfnrosHXHXueVz/1zlcfe55iKR1D84RwG9dF5HuNHRdETkOu2KvakMtcAbwe6Dzfq++gO1y/BhYjA3hnUAJsJDfX/LsqvycXD7eUs3QkjK+deO1TL/uF7y/YV3c6k8wl+APnO66iHSmoeuCnVNhHjoJeZsasYE7A5jW6hZzvVcEOAQ7BcHKL149bmRN36vvv5sbzv8+f3jyMWZMmsx13/ke1z3451iXnsgewB8ocV1EutLQdWMO2o/bJgOcj+3LPfCYpxLsrJcAn2JHLQzYe4QdGz57o6ioR08GFZewe88efD4fGRk+du9J627NbsAj+AP6798BvZAWbyLnc6BGm/qK17Dj6A7DdiCAHbe0HoC7M2y/7TXAd7ytDPBroAcAXTs2rvvVQw+MfHz2zQBcdNrXmfGra2hqbuaey9PuBqt9jcO+gWl7ZdEVMSaVpxlNMCK9sJ9902XylZgRgvUgOQfb5ugRta+8/seVx8erpiS0ExhGZcXHrgtpTVVV1dry8vItrutoS1VVVY/y8vKy9m6vLd34+j0auHFz7GG1aT1UoR06A38AznRdSHuIENWpHY2hzXG/1113Xc+HHnqoUEQYOnTo7scee2xtfn5+RC1V7dOJF5HJ2DWtVJxMHLMz1Scuj4Yz8Aemui4iEa1Zsybrvvvu67V48eLlH3zwwbLm5maZM2dOQaTH1dCNB5F84G7XZaSbo4bvSoeJy6PhLvwBvSOyFc3NzbJr1y5fY2MjdXV1vuLi4sZIj6mhGx+zsGOZVJxkZphNXTs2a0u3fUqB2a6LSDT9+/dvvPjiizf3799/ZM+ePcs7derUPG3atJ2RHldDN9ZEBnOwEU8qJnoXpN3E5ZG6DH9Al4Zqobq6OuPZZ5/t+uGHH763efPmJbt37/bdfffd2r2QBH6NXrCMu1GH7N7tuoYkk4l2gX3FM88807mkpKS+b9++TTk5Oeb000/f/vrrr3eM9LgaurEkciygt1w6MH5UTbbrGpLQOPyBya6LSBRlZWUN77zzTseamhpfMBjkpZde6jRs2DCd2jHB3eq6gHQ1YXRNVJYiT0OzgH+5LqI17RniFU0TJ07cddppp30+cuTIYZmZmYwYMWL3FVdcUR3pcfXmiFgRmYqd5FXFwMFvjjANDf9e5MvK1EZFmE6hssJ58KbqzRHavRALdv5AXXrHkfyc4FoN3IjMdl1AKtPQjY2p2PVjlAP9+zRE/BEwzR2JP3CK6yJSlYZubFzpuoB0duSw2ogHsKuEaO0Gg8FgQt/K7dUXDGUfDd1oEzkc0ElWHJowpibiYT2KI/AHpjiuYWl1dXWXRA3eYDAo1dXVXYCloeyn/V7Rp61cx44fWVvkuoYUcQ3wnKuTNzU1XbB58+Y5mzdvPpTEbCAGgaVNTU0XhLKTjl6IJpH+wAfoihAxd6DRC4LZHly4KOYzuW34bDPfvmk2m7dtxecTLpr6dX505je55v57eOq1V/CJ0LNbAX/5+Sz69ij8yr4vvxvg8rtu/+L7levX8ei1N3L6ceOZ8atf8t7qj5h69LHcdOHFANzw1zmMHDCIrx1bGesfqzWjqKyocnHiVJWI/3sks8vQwHWqW+fmuCx+lpmRyW0/uIwVf/0b/7t7Ln/8x3yWr13NT87+FkseeITF9z/M1KOP5foH5+y374TRFSy+/2EW3/8wL/3uHvJzcznp8KNY8tEHACx54BH+u2QxO2pr+WTrFt5ascxV4AKE1IpTbdPQjRaRPOwSBsqh4aV12+Nxnj7dezBm8FAAOuV3YFhpGZu2VNO5w5fdybv21LW5+vB8/3845cijyc/NJSszk7r6eoLBIA1NjWT4fFz7wL1c/93vx/RnacMM/IFclwWkGu3TjZ5ptLZYrYqrY0fWxr0hsfaTj3n3g1UcOWwEAFfPuZu/vvAsXTp05OXf33vQfR996UWuOOscAIaV9qekZ2/GXHgu3zppCh9u2oAxMHrQkJj/DAfRDbs26DyXRaQS7dONFpEFwImuy0gXB+rTXfDb95eeWLHz0HjVUbt7N5WXfY+rzz2PacdP/MprN8+by56GBq4773ut7vvJ1i2M/O43+fiJ58nK3L/9c9ovLudPP76Kuc8/Q9VHH3BixRFcOPXrMfk52rCQyooJLk6cirR7IRpEioETXJehjDlqeG1ZvM7W2NTEGbN+xoxJk/cLXIBzTpjME/6XDrj/4y+/yNePG99q4D71qp+KIcPZtaeOpWs+4vHZN/PQgufZvSfi+VbCUYk/cIiLE6ciDd3o+Db6XjqXmWE2dsoPxmWMrjGG82+9gWElZVzxjRlfPP/BxvVffP30668wtKTsgMd45D8L+OYJJ+/3fGNTE3c88Sg/Oftb7N6z54t+4WAwSEOjk/s+BDjfxYlTkfbpRsdM1wUo6Nuj8ROgXzzO9dp7VTy04DkOG3AIo863fbI3XXgx9z/3FKvWr8Pn81Haqzf3XvELAAIrl3Pv008y56e/BGw/8IbqT6ksH7Pfsf/498eZefKp5OfmMnLgIIwxHHbe2Uw56hi6duoUjx+vNTPxB66hsqLJVQGpQvt0IyUyBuI75ZxqvU/39GM/X/j3X3003lFJ6eAEKisO3F+i2kU/EkfuNNcFKGv8qJoDTPWoosT1bcEpQUM3chq6CaJyVE0v1zWkOJ15LAo0dCMh0hfYv1NOOWD2HNq/Tpdcj63h+AP6HkdIQzcyp2Kv7CrHOuQG12Zm6C3YcaCt3Qhp6EZGuxYSxIC+9Vtd15AmtF83Qhq64RLJRm+ISBhHj9ilQ5niYyL+gF6wjICGbvjGAPmui1DWhNE7deLy+OiATtIfEQ3d8I1zXYD60rGH1cblpggFaL9uRDR0w3eM6wKUJWK2Fhc29nRdRxo50nUByUxDN3xHuy5AWd07N61veysVRSPxBzQ7wqRvXDjssjx9XJehrBFle2pc15BmOgI661iYNHTDo63cBHJ8eY2OlY6/Ua4LSFYauuEZ7boA9aUJo2sKXNeQhvTfQJg0dMMz1HUBai8TPHzorjLXVaQhbemGSUM3PMNcF6CsrEyzoWNesIPrOtKQtnTDpKEbKpEcoMx1GcoqthOXq/jrhT/Q23URyUhDN3SDQSdWSRRjhuyqd11DGit3XUAy0tANnfbnJpAJo3XicodKXBeQjDR0QzfIdQHqS5XlOnG5QzpWPQwauqHTX7SEYeqGle7RSbXd0T7dMGjohk5bVgmiY15wTYZPf4cd0tANg/7Chk5DN0EcUlS/zXUNaU5DNwwauqHT0E0QR4+obXZdQ5rT0A2Dhm7odArBBDFxzM5OrmtIcxq6YdDQDYVdoqeb6zKUpROXO5eHP9DFdRHJRkM3NNqyShCZPrOld0FToes6lHa3hUpDNzQ6ED9B9OrWsNF1DQqAXNcFJBsN3dBkuS5AWeUDd+9yXYNS4dDQDU226wKUNX5Ujf4HqJKShm5o9B96gpgwuqa76xoUALpqR4gyXReQZLSlmyC67Xp+3Ts7e2/pkd0tr2tmpw75GTldM8joIXaEiVIJS4wxrmtIHiIVwNuuy1DW2i58ctkprHp6CIcaoQdA98wu24tzem4ty+tbMzC3ePeAvKLGstw+FOf0zCzM6pbTObNjpzxfdjcfvu4iolN0Rm40lRWLXReRTDR0QyFSDugvWILZk0H97eMI3HIs3WpyGN6efXxIsDCrYGtJbq/tA3KLavrnFdUNzC1qKs3tQ9+cwsweWV3zO2d06JTjy+omSIGI6Mfo1mnohkhDNxQig4FVrstQB/ZSf5b98BS2LyvkcCQ63UGZktHUJ7vHltLcPtv75/atGZBXVD8gt6i5JLc3fbN7ZHfP6pLfMSO/c5ZkdfeJdI7GOZPIGCor3nVdRDLR0A2FSD9gvesyVNs+7kj1lSez/PERDG72xW86zlxf9p6+2YVbSnP77BiYV7RrQG5Rff/comBJbm/pnd09p1tm5/wOGbldsiSzh4jkx6uuGNLQDZGGbihEugE6s1USafTRdPfhvH19Jfnb8hNreZmOGXm1xTm9tpXl9tkxILdo14C84ob+uX1Nv5yevp7Z3XO6ZXbqmOfL6ZIpGT3Ers2XiIZQWfG+6yKSiYZuKOyFlybXZajw/K+IVZecymeL+lCBkOe6nlAUZHbe0S+n19bS3D47B+YV13kXCIPF2T0zC7MLcrtkduiY58vp5sPXI84XCLtTWaENkRBo6IZKpBbQJb+T2JY8Pv/5JJY8OIoBTRmk1KQ5PiTYI6vbtpLcXp/3zy2qGZhXVDfAu0BYlFOY1SOra26njA6dcn3ZBVG4QNgMZFFZoSESAg3dUImsQxfkSwnNQvAvowhcfQIZn3ZkrOt64i1TMpp6Z3ffWprT5/OyvD61A3OL9wzIK2ouyelN35zCLy4QZktWgU+ktdnEtlBZoZMOhUhDN1QirwHjXJehoquqF6svmcLGV0sYjehscvvKkaz6opyeW0pye28fkFu0a2BecX1RTuHHM6fPPtt1bclGQzdUIo8C012XoWJjZzY1107k3Xsq6NeQSX/X9SS4/5hZZpLrIpKNzr0Qug2uC1Cx07mBTr//F8fX/YqyR/7Gon47eBtDMKonCQL3AvNaeW078CBwNzAX2OE9vwX4E3APX/4GNnvbNkS1ulBscnbmJKahGzoN3TTgAzl7GWPX/47DV97FxhM/wi+G7VE5+P/A3rTcigVAOfADoBL4j/d8AJgEfAN4vcVz5bicEURDNwwauqHT0E0zQ7ZSsuAhKmtuIvuqV/hvXiPhj0vdAXwAjDnA69XwRadGf2Cl93UG0Og9fEAd9t5ItyOP1zk9e5LS0A2d3pGWpjo0kn/jSxy3+0YGPzOPJYds5Q1MiOO2/wWcyIEnROwFrPC+XoHtOtgNHA68AfwTOA7wA8cf5Djxsczp2ZOUTu0YulWAwfWvu3Jq6geMnPqBN9PZZN5/ZgjDgz4OPnxqFXaEd19gzQG2OQl4DjutUil2VT4f0BU4z9tmK1CD7aJ4Etu3O4EDd1nEztK4nzEF6OiFcIh8BAxwXYZKHHsyqP/d0QRuPo6uNTmMaHWjfwNV2BBtAuqBYcAZBzhoPXAX8ON9nv8bMBEbzCXYQH7lIMeJjU1mlimO6xlThLZ0w1OFhq5qIbeZnF+8yjG/eBVeLmPZpVNamelskvcA29J9nf2DcheQhw3mV4HR+7y+Ftv67Y7t3xVv28bo/jztoF0LYdLQDU8V8HXXRajENGEtI5beDZs7UP3jk1n+2KFtzHT2ErbLYSg2VPeOWCgFTm2xncG2aM/yvh8LPIEdgjY1qj9Ce2jXQpi0eyEcIqcDf3ddhkoOjT6a7q0gMHs8eYk201kEvmtmmbmui0hGGrrhEOkPrHZdhko+b3yaF9sAAAr2SURBVBXx/sVT+DTQl7EIyTyf7hFmltGlq8KgoRsOOzPTFqDAdSkqOW3NY/svJlE1NzlnOtsDdDGzjLt74ZKYhm64RP4OnO66DJXcmoXgg+UErppE5qcdGI0kxVBEv5llxrsuIlnpzRHhW+i6AJX8Mgy+7y7miM2/ZcySe1h73DpewVDjuq42/Le9G4pIPxF5WURWiMgyEflRLAtLBtrSDZfISOwoBqWiamc2NbMm8M7dh1OSoDOdnWRmmRfbs6GI9AH6GGPeEZFOwCLgdGPM8phWmMA0dMNl+3U/w8V9QCotBMHMH8E7PzmR5vVdqEAS4pNpPdDNzDJ14ewsIk8BdxnTvtBORRq6kRCZT7zvA1Jp6f0CNlw6hdUvDqTcCF0dlvKymWUmhrOjiJRhRxofaozZGc2ikkki/M+ZzF5yXYBKD4O30e+F/6Oy9kZyrvZHONNZZP4dzk4i0hF7K8dl6Ry4oC3dyIiUoNPbKUeeHUTV5ZPZ/UEBhyNxu7t0lJllQrqWISJZ2PnRXjDG3B6bspKHhm6kRN7CTrwXd9uBC7D3YwrwAPa2/e9jB1JmYhcgOKKVfdd7+27w9n0OKANmAO9h7yq9ydv2BmAk8LXY/BgqQus788llp/D+U+2Z6Swy75tZZkgoO3irDT8IbDPGXBabspKLdi9Ebr6rE/8ImIyd57oKO2HVT4FZ2Amorve+b823gZ9gp2x9C+gJLPFeW4IdE7QD+MR7XQM3cZXspM+Tj1G5+0a63Pwir3XeE7PJaP4Wxj7HAN8CJorIYu8xJcp1JRWd8CZyTwC/jvdJd2KvSPzF+z7be4j3GtjQ7NvKvsuxMwue6H3f0fszC7sgQRA7d3YGcC02vFXiy2km++evcczPX4OFpSy/dAqfL+1JBUJOlE7xeKg7GGNeReee/grtXogGkXeBUfE85WLgImA4tpU7FrgD221wMnZCqiB29sDSffb9BzAHG9JrsLMN3oIN2cuwd318CzgBO53rnJj+JCqWNneg+sqTWfbooQw56ExnbVtpZplhUSssjWnoRoPIL7Fdn3ETAI4CXgOOxHY1dMa2biux49geB+5j/8vN84HzgXexc2BPB6Z4z7V0GnYB2rnYYD8RuDD6P4qKg0YfTX+q4O1Z48kPc6azG8wsc23UC0tD2qcbHfOwjcu4KfYeR3rfnwm8g71iMc177ixsf2xr+47GzsKeiZ1A4p19tnkKqMDOqb0UG+APYZfrUsknK0jmJW9x9NZbKX/zz7x/+Cb+iwnprzPkrgXVOg3daDBmDXbx7LjpDfTDLrsFdt7r4dg+XL/33EvAoFb2PRz4HLvw7N7thrd4vRHbVfETbMju7ZDb29erktsRmxj81p85bsutNFwUwJ/V3OZiq0vMLKOTlkeJhm70/CneJ7wTO8RrJLaP9yrgz9gltcq97+/ztg1gh4iB7bv9LbbP9jBsE71lt8EfgZlAvnds4213DDi9FUpFV/c6uv7pn1TW/YriB/7BW71rWIRp9RPbvXEvLoVpn260iGRir2NFcrFCKafe68maS6aw4ZVSRiF0BmqBvmaWSfSZz5KGhm40idwA/NJ1GUpFamc2NdeN592/jeCN9bebn7uuJ5Vo6EaTvS14Ddpto1LHCNJ4GsZY0HCIJmPWYy/8K5UKXtTAjT4N3ei70XUBSkXJ710XkIq0eyEWRP6FvTFMqWT1LsaMcV1EKtKWbmzE9e40pWJgtusCUpW2dGNFZCH2jlylkk0AY5xMV5oOtKUbO9q3q5LVLNcFpDJt6caSiB843nUZSoXgTYw5ynURqUxburF1OXbKAqWShbZyY0xDN5aM2Tvxl1LJ4DmMecF1EalOuxdiTaQP8D5fLtCgVCKqBw7FmA9dF5LqtKUba8Z8gl2YQalEdpsGbnxoSzceRHKx60fuu3KOUolgAzAUY3SO+jjQlm48GLMHuMR1GUodwI81cONHQzdejPkn8KjrMpTaxwKMCWdpdRUm7V6IJ5FC7AroPVyXohSwHTgMYza6LiSdaEs3noypBn7gugylPJdo4Mafhm4ERGSyiKwSkQ9FpH2z69uPco/FtjKl2vQ3jJnnuoh0pN0LYRKRDOz42xOBjcDbwDdNeyZ9FikAqrCroSsVb5uxY3K3ui4kHWlLN3xHAB8aY1YbYxqwF8m+1q49jdkGTAeaYleeUgd0vgauOxq64SvCjm/ca6P3XPsY8zrwsyjXpFRbbsWY51wXkc40dMMnrTwXWl+NMbcDT0alGqXa9m/gKtdFpDsN3fBtBPq1+L4Y+DiM43wX0NsvVaytBc7GmGbXhaQ7Dd3wvQ0MEpH+IpINnA08HfJRjNkBnAnoHUEqVuqAadqPmxg0dMNkjGnC3tr7ArACeNwYsyzMg1UB56Bz76rYuAhj3nVdhLJ0yFgiEbkEuNN1GSql3IQxV7suQn1JQzfRiPwGuNJ1GSol3I8xF7guQn2Vhm6iERHsmN9vuC5FJbWngDP0wlni0dBNRCI52L5iXcJdheO/wEnelKIqwWjoJiqRjsBzwHGuS1FJZSlwHMZsd12Iap2OXkhUxtQCU7CtFqXaYwlwggZuYtPQTWQavKr9FgETMOYz14Wog9PQTXQ2eE8BXnFdikpYr2NbuNtcF6LapqGbDIzZhW3xvuC6FJVwXsZeNNvhuhDVPhq6ycIG71TgftelqITxT2CK97uhkoSGbjIxpskb7H6N61KUc7cDX9NhYclHh4wlK5EZwANAtutSVFw1Aj/AmDmuC1Hh0dBNZiLjgSeAAseVqPjYhr3LbKHrQlT4NHSTnUgJ8DhwpOtSVEytAqZijM69nOS0TzfZGbMee9faHa5LUTHzMFChgZsatKWbSkTOwPbzdnZdioqKOuBHGPNn14Wo6NHQTTUiA7GzlFW4LkVFxE5sb8xy14Wo6NLuhVRjzEfA0dgFCOsdV6NCF8QOBztCAzc1aUs3lYkMx3Y36EW25LAEu7TOm64LUbGjLd1UZltK47ArUdQ5rkYdWB3wC2CsBm7q05ZuuhA5BPgd9lZilTheBL6PMatdF6LiQ1u66cKYDzHmNOBE4D3X5ShWA9/EmJM0cNOLhm66MebfwGjge4DOvRp/1cCPgGEY86jrYlT8afdCOhPphO1LvBTo6LiaVLcL271zK8bUuC5GuaOhq0CkO/Bj4BKgk+NqUs0uYA5wC8Zsdl2Mck9DV31JpBvwA+CHQE/H1SS7T4E/APdgzOeui1GJQ0NX7U8kD/gO8H1gpNtiks5K4DbgIYzRm1PUfjR01cGJHAVcBEwH8h1Xk6jqgX8Ac4EF6D8qdRAauqp9RDoDM4ALsaMfFLyNDdpHtQtBtZeGrgqdyGDgDO8x1nE18bYSeArbfbDMdTEq+WjoqsiIlALTvMc4Um/sdxPwKvAM8LTOaasipaGrokekK3A8MN57lJN8IWyA5cBrwELgX9p1oKJJQ1fFzpchfBw2gEcCvZzWtL9aYBE2ZF8D3tCQVbGkoaviS6QnNnz3Pg4BioG+QFYMz1yN7Y9dgW3JrvAeG3W0gYonDV2VGER82FZwMV+GcCegQyuPLKDBe9Tv8/XnwBZsyH4KbAI+xpg9cfxplDogDV2llIqjZLvIoZRSSU1DVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4khDVyml4uj/AYpodSt1RgsdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Figure 2 probability of corresponding numbers [0-9]:\n",
      " [-5.470294    8.598144   -0.29099795 -1.5832896  -0.88322777 -1.8221556\n",
      " -0.637848   -0.71524227  2.496806   -0.6640963 ]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD3CAYAAAC+eIeLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAdiklEQVR4nO3deZhU5Zn+8e/TNKAoi0KDAiq4gRsuxA0dg8skuOAWzaYmbpPRmLjEmJhNLDP+ktFMiJOJcfw5xglxnSyjxiEXrsm4EOPCoMQFFaVZWpodRKCbfuaP9/TQtA10VZ2qt07V/bmuurqp5dRNQd99znuW19wdEREpj7rYAUREaolKV0SkjFS6IiJlpNIVESkjla6ISBmpdEVEykilKxXFzJ4ys4tLtOxdzWy1mfVI/jzEzP5kZqvM7J/M7NtmdkcJ3vccM5uW9nIlm+pjB5B0mdm7wMXu/ljsLJXG3ecC23e460vAYqCfp3TAupmNAOYAPd29NXnfu4G701i+ZJ/WdKWW7Qb8Na3CFekOlW4VM7PzzewZM5tsZsvN7B0zG5fc32hmi8zsix2ef7KZvWxmK5PHr++0vC+Y2XtmtsTMvmdm75rZCcljdWZ2rZm9nTz+gJntuIVsp5nZjOS93jazCV08Zw8zeyJZ3mIzu9vMBnR4/JtmNj8ZHnjDzI5P7j/MzF5Ilv2+mf04uX+EmbmZ1ZvZXcAXgW8kQw4nmNn1ZvarDss/2syeTT67RjM7vxuf05+Sr8uT5R6ZfN5Pd1juODP7i5mtSL6O6/DYU2b2/eTfbZWZTTOzQZv/V5asUelWv8OBmcBA4B7gPuBQYE/gXOBfzKx9k/sD4AvAAOBk4FIzOx3AzPYFbgXOAXYG+gPDOrzP5cDpwMeBocAy4GddBTKzw4BfAtck73UM8G5XTwV+kCxvH2AX4PpkGaOArwCHuntf4JMdlnELcIu79wP2AB7ovGB3P5+wyX+Tu2/feTjGzHYFpgI/BRqAg4AZW/uckr8LwIBkuc91Wu6OwCPAPxP+TX4MPGJmAzs87fPABcBgoBfw9S4+G8kolW71m+Puv3D3DcD9hOK6wd3Xufs0YD2hgHH3p9z9FXdvc/eZwL2EEgU4C3jY3Z929/XAdUDHzfK/B77j7vPcfR2hHM8ys672G1wE3OnujybvNd/dX+/8JHd/K3nOOndvJhRUe54NQG9gXzPr6e7vuvvbyWMtwJ5mNsjdV7v79AI+t3OAx9z9Xndvcfcl7j6jG5/T1pwMzHb3Ke7e6u73Aq8DEzs85xfu/qa7f0j4hXFQAfmlQql0q9/7Hb7/EMDdO9+3PYCZHW5mT5pZs5mtAC4B2jdthwKN7S9y9zXAkg7L2Q34XbIpvhx4jVCMQ7rItAvwdhf3b8LMBpvZfckQwkrgV+153P0t4EpCuS9Knjc0eelFwN7A68nm+ylbe698Mm7lc9qaocB7ne57j023Gpo6fL+GTXf+ScapdKWje4CHgF3cvT9wG2ETH2AhMLz9iWa2LWHzuF0jcKK7D+hw28bd53fxPo2Ezf6t+QFhbXpMMlRwboc8uPs97n40ofAd+Mfk/tnu/jnC5vk/Ar82s+268X7dzbilz2lrO+UWJHk72hXo6nOSKqTSlY76AkvdfW0y7vr5Do/9GpiY7ATqBeToUICE4rnRzHYDMLMGMzttM+/zb8AFZnZ8sgNumJmN3kye1YSdUsMIY8Akyx9lZseZWW9gLWGNfUPy2Llm1uDubcDy5CUb8vokwnjvCWb26WTH20Aza9/M39Ln1Ay0AbtvZrn/BextZp9PlvsZYF/g93nmk4xS6UpHXwZuMLNVhDHb/9sB5e6zgK8SdsQtBFYBi4B1yVNuIaz9TUteP52wE+8j3P15wo6iycAK4I98dO0PQrEfkjznEeC3HR7rDfyQcJxtE2Gt9tvJYxOAWWa2Osn1WXdf290PIck4FzgJuBpYStiJdmDy8JY+pzXAjcAzyVDLEZ2WuwQ4JVnuEuAbwCnuvjiffJJdpkMUpRDJEQ/Lgb3cfU7sPCJZoTVd6TYzm2hmfZLx0R8Br9D1oV4ishkqXcnHaYQdQQuAvQib7dpUEsmDhhdERMpIa7oiImWk0hURKSOVrohIGal0RUTKSKUrIlJGKl0RkTLSdD0iUpFefPHFwfX19XcA+1OZK4htwKutra0Xjx07dlF3X6TSFZGKVF9ff8dOO+20T0NDw7K6urqKO6Ggra3Nmpub921qaroDOLW7r6vE3x4iIgD7NzQ0rKzEwgWoq6vzhoaGFYQ18e6/rkR5RESKVVephdsuyZdXj6p0RUTKSGO6UhnMehKuqTuYMCPFoE5fBwJ9gB5Aj+vGM+f749kdaE1uLYRr/L6f3BZ1+P59YJFP8vXl/CtJyszGpro89xe39pSzzz57xOOPP95/4MCBrbNnz56VxtuqdKW8zIYAozrdRgMjCYXaLWt60oONM+92761ztpQwCeTMDrdXfJKvzGc5UjsuvPDCxVdcccWiCy64YGRay1TpSumEmYAPAo4GjkpuO0dMtCMwLrn9H8vZe2ws4b8AT6qIBeDEE09c/cYbb/RKc5kqXUlPGCL4G8J05EcTpuvJd0LIGHZLbu3ToLdazv4MPApMA573SZ7vHGsiXVLpSnHMdiDM+TUR+CTQL26gVNSzcc38emCF5ewJQgn/wSdpeiIpnEpX8mc2EDgDOBs4FugZN1DJ9Sf8fc8AsJxNB34J3OeTfFnMYJI9Kl3pHjMDjge+BJxO9RftlhyR3CZbzn4P/Dsw1Sd5a9xYkgUqXdkys8GE6dIvBvaMnKbS9AY+ldyaLWf3AHf5JJ8RN1aV6sYhXmmbOHHiyOnTp/ddtmxZ/ZAhQ8Zce+21C6666qrFxSxTpStdMxsPfBmt1XZXA3AFcIXl7CngBz7Jp8WNJMV6+OGHUx+/V+nKpswmAN+j02FVkpfxwHjL2UvAD4Hf+CRvixtJKoVOA5bAbCJmzwNTUeGm5RDgAeA1y9lFlrNUj/eUbFLp1jIzw+xMzF4CHgIOjR2pSu0N3AG8bTm7wnKm4ZoaptKtVWaHA38GfgMcHDlNrRgO/AR4xXI2odRvZmZXmdksM3vVzO41s21K/Z6ydSrdWmM2BLNfAM+hNdtYRgFTLWcPW872KMUbmNkw4HLgY+6+P+G6Fp8txXtJflS6tcKsHrOrgDeA8wGLG0gIZ/LNspz9wHJWitOl64FtLVwDow+woATvIXnS0Qu1wOw44KfAvrGjyEf0Bq4FzrOcfdMn+d1pLNTd55vZj4C5wIfANPdsH8JmuXQv7eiTtn7cby6XGzxlypQGM2P06NFr7r///nf79OlT1IXVtaZbzcz6YnY78Dgq3Eo3DPiV5Wyq5WynYhdm4ZoYpxEumTkU2M7Mzi12ubVkzpw5PW+//fYhM2bM+Ovs2bNnbdiwwe64444di12uSrdahZMbZgJ/FzmJ5GcCMNNyNnGrz9yyE4A57t7s7i3Ab9GhgHnbsGGDffDBB3UtLS18+OGHdcOHD28pdpkq3Wpj1guzm4EngBGR00hhGoCHLGc/t5xtW+Ay5gJHmFkf23jdjNdSS1gDRo4c2XLZZZc1jRw5cszgwYMP7Nu374Yzzzyz6Ossq3SridkowlEJX0c7yqrBJcBLlrO8D+lz9z8DvwZeAl4h/Kzfnm686tbc3NzjkUceGfDWW2+90tTUNHPNmjV1t956q4YXJGF2FvAi4SwoqR6jgemWs2ssZ3n9InX3Se4+2t33d/fz3H1diTJWpYcffrjfrrvuum7o0KGtvXv39tNPP335s88+u32xy1XpZl04q+z7hNNNszBLg+SvF3AT8LDlrG/sMLVixIgR61966aXtV61aVdfW1sYTTzzRd5999llb7HJ1yFiWmW0PTCFcCUyq38nAc5azU32SvxM7TLl15xCvNB133HEfTJw4cdmYMWP2qa+vZ7/99lvzta99rbnY5ap0s8psd+BBYP/YUaSs9gOet5x9yif5H2OHqXaTJ09eMHny5FRPKtHwQhaZHQs8jwq3Vg0EplnOzokdRPKn0s0aszOBPxB+8KR29QKmWM6+FTuI5EelmyVm5xF2mOm6rALhsMD/Zzm71XJWjT/LbW1tbRV96GOSL68L1FfjP1R1MruEMAFij9hRpOJcCtyR7yFlGfBqc3Nz/0ot3ra2Nmtubu4PvJrP67QjLQvMvg7cHDuGVLQLgBbL2SU+yYu6IEulaG1tvbipqemOpqam/anMFcQ24NXW1taL83mRSrfSmeWA62LHkEz4EtACfCV2kDSMHTt2EXBq7Bxpq8TfHtLO7DuocCU/l1nOJscOIZun0q1UZhcB/xA7hmTSlZazm2KHkK6pdCuR2UTgX2PHkEy7xnJ2Y+wQ8lEq3UpjNg64Hx2lIMX7tuWsKsZ3q4lKt5KY7Qs8DBR6DVWRziZbzo6LHUI2UulWCrOdCWeaFX29TpEO6oH/sJztHjuIBCrdSmDWC/gNsEvsKFKVdiTMRKHLQlYAlW5luAU4MnYIqWr7ESa+rMizu2qJSjc2swsI07KIlNqpwPdjh6h1Kt2YzA4AfhY7htSU71jOPh07RC1T6cYSZn34D3SkgpTf7ZYz7T+IRKUbz63AqNghpCb1B+7U+G4cKt0YzE4FzosdQ2raCcCXY4eoRSrdcjMbANwWO4YIcJPlbM/YIWqNSrf8JgM7xw4hAvQB/r1KZ52oWPqwy8lsAnB+7BgiHYwDvh47RC1R6ZaLWV/g9tgxRLpwg+VMM0uXiUq3fG5Cp/lKZeqNjhcvG5VuOZgdTJhKRaRSHWM5Oz12iFqg0i2Pm9FnLZXvJstZz9ghqp0mpiw1s5OA42PHkDytAH4HrAYMGAscAUwD3iBcYn5H4DS6PqdwMmGj3Qi/bv8+uf9RYDawE3Bmct//AB8my49rL8Kxu7fEDlLNtPZVSmY9CGO5kjV1wCcI8+peDDwPLAJ2J9TSl4GBwNNbWMYXgUvZWLhrgcbktQ68T5i7dwZwaOp/g0JdZznbIXaIaqbSLa2LCJfUk6zpCwxNvu8NNACrgD3ZOJHScGBlHss0YAOhcFsIP33PAIdTSZMz7Qh8N3aIaqbSLZVwQZtc7BiSgmXAQmBYp/tfJpRwVwyYQphe9IXkvt7APoTzEXcAtgEWAKNTzlu8r1jO9ogdolppTLd0riSM3EmWrQMeACYQSrLdnwirLGM287oLgX6EMeEpwCBgBHB0cgN4EDgWeBF4GxgCfDzV9IXqBdwIfDZ2kGqkNd1SMNsWuDx2DCnSBkLhHgDs2+H+GcCbhB1hm7tOV7/k6/aENdn5nR5fmHwdSNiR9mnCmPGSolOn5SzNq1YaKt3SOJ8wCihZ5YQ10UGEE2XbzSbsPPscYX2wK+sJa8jt378NDO70nCcIa7ntY7wQCryl2OCp6UHYWpOUaXghbWZ1wNdix5AizQVmEsry58l9xwNTCUX5y+S+4cBEwg61h4BzCUMK9yePtxHWlPfqsOzXCOPD/Tos41bC8EJlDUhdaDmb5JN8Wewg1USlm74z2PzuFcmK3YDru7h/7808vx+hcCHs/790C8veJ7m1+2S+4cpmO8IBbz+MHaSaaHghfd+IHUAkRV/VWWrpUummyewY4LDYMURSNJQwgi0pUemm64rYAURK4OrYAaqJSjctZg2EXSoi1WaM5Wx87BDVQqWbnvMAjX1JtTondoBqodJNzwWxA4iU0JnaoZYOlW4azA4CNN2JVLMdqeSD2zJEpZuOz8cOIFIGOoohBSrdYoUz0PSfUWrBaZazPrFDZJ1Kt3hHEE7kFKl226EjdIqm0i3eSbEDiJSRtuqKpNIt3omxA4iU0QTLWb+tP002R6VbDLMhwMGxY4iUUW/gmNghskylW5wJbP4y1iLVanzsAFmm0i2OhhakFo2PHSDLVLqFCtOrfyJ2DJEIDrac9Y8dIqtUuoUbS5jTVaTW1KFx3YKpdAt3eOwAIhGNjx0gq1S6hftY7AAiER0bO0BWqXQLd2jsACIRHWg5GxA7RBapdAthtj0wKnYMkYjq0NRUBVHpFuYQ9NmJ6HKmBVBxFEbjuSJwQOwAWaTSLczY2AFEKoDWdAtQHztARmk8V2qTs7ZPC3OHr2TxYfP5ADPD3WPHyhKVbmFGxA4gUkrmLB2wlnl7LGXFIQtpO2oufQ6fz057LmVYD2dvYO/kqcOAeRGjZo5KN19mfYGBsWOIFM3x+jbmD/6Apn0W88Hh86gb10j/Q+czbPAaBhLmRduavVDp5kWlm78RsQOI5MVZu20rjcNXsnhME+vHzaPnkY0MOrCJXfq0MpziZj7ZC3gypaQ1QaWbvxGxA4h0xZxl/dcyb/dlLB/bPiQwjyF7LWV4D2cvQkGmbbcSLLOqqXTzNyJ2AKlhyZBAwxreH72Y1YfNo+7oRvp9bD7DdvqAQZT/IkwNZX6/zFPp5m9k7ABSA5x127Yyd9hKlhzwPmuPnEevoxrZ8cAmdt2upeghgTSpdPOk0s3fsNgBpHqYs7zfOubtvoxlhyyg7ahGtj1iHjvttYRh9aUbEkiTSjdPKt386Rq6kp8wJLBg0BqaRi9m9WHzqTtqLn0PXcDwnVczCMjyhWMGxQ6QNSrd/GX5B0RKyVm/TStzh61icfuQwLi57HBQE7tu38IwqnMrSWu6eVLp5k/TlNQ6Z0X/dTSOCEcJtI5rpM8R8xg8ajG71Dt7AnvGjlhGO2BWj3tr7CBZodLNX9/YAaQ8eiRDAqPCkABHNdLv0PkMHbaKweiXbzsjDLk1xw6SFSrd/G0bO4CkyGnZppW5Q1fRfMAi1h7RSK+jGtnhoCZ26bueocDQ2BEzoGfsAFmi0s1fn9gBpADOyn7raByxnOWHLKRlXCN9jmykYdQSdunZxh7AHrEjZpjFDpAlKt18mBnQK3YM2bwebTQNXMOCUUvCkMC4ufQ7bD47D1/FEGC/2PmqlC4RmweVbj7cHTNHv9mjG7GCniOXMX3/Raw7spH6IxvZ4ZCF7NJvPTsBO8XOV2P085AH06Uw82S2FugdO4ZIBRmB+3uxQ2SFNgvyty52AJEKozXdPKh086fSFdmUSjcPKt38qXRFNrU+doAsUenmT6UrsqmlsQNkiUo3fypdkY3W4v5h7BBZotLN36rYAUQqyJLYAbJGpZu/hbEDiFQQDS3kSaWbv/mxA4hUEJVunlS6+VsQO4BIBdHwQp5UuvnTmq7IRotjB8galW7+tKYrspFO/82TSjd/Kl2Rjd6OHSBrVLr5mxc7gEgFUenmSaWbL/cVaG1XpN1bsQNkjUq3MDNjBxCpAE24L48dImtUuoVR6YrAa7EDZJFKtzD/EzuASAVQ6RZApVsYremKwMuxA2SRSrcwr6NriIo8EztAFql0C+HeCvw1dgyRiJYSVj4kTyrdwj0fO4BIRNPRrLYFUekW7o+xA4hEpKGFAql0C6fSlVr2bOwAWWXaQiiC2ZvAXrFjiJRZK9Af9zWxg2SR1nSL83jsACIRvKDCLZxKtzjTYgcQieCh2AGyTMMLxTDrT7iIc33sKCJltB/uOmSyQFrTLUa44pj24koteUuFWxyVbvHujx1ApIwejB0g61S6xfs1YW+uSC1Q6RZJpVss92bgidgxRMqgGQ2nFU2lm477YgcQKYOHcG+LHSLrdPRCGsJRDIuAXrGjiJTQONyfix0i67Smm4ZwFMMfYscQKaFZKtx0qHTTMyV2AJES+v+xA1QLDS+kxawemAMMjx1FJGXrgKG4L40dpBpoTTct4cLmP48dQ6QEfqPCTY/WdNNkNghoBLaJHUUkRcfi/lTsENVCa7ppcl8M3BM7hkiKXlfhpkulm75bYgcQSdGPYgeoNhpeKAWzp4CPx47R7g3gMx3+/A5wA/Bc8hjAcmAAMKOL148A+gI9CJdTeyG5/5vAVOAg4JfJfVMIMxZekVp6iWg+sDvumvk6RbokYWncTAWV7ig2lukGYBhwBnBlh+dcDfTfwjKeBAZ1+PMKwnwtM4FzgFeAPYG70AHLVeTHKtz0aXihFNwfoULnkHoc2APYrcN9DjwAfC6P5dQB65PXfgj0JPymuTz5XjKvGbgtdohqpNItnW/HDtCV+/houf43MITNT/ZmwCeAscDtyX19gU8BBwMjCWvJfwFOSzmvRHOzpuQpDY3plpLZNOBvY8dotx4YCswilGy7SwlDA1dv5nULktctIvxlfgoc0+k5FwOXAS8S5jAaA3w3reBSbs3ACJVuaWhNt7Qqam13KnAImxZuK/BbNt3R1tnQ5Otgwljw850efzn5ujdhh9oDwKvA7CLzSjT/oMItHZVuKbm/QOi0inAvHx1aeAwYzebPXf4AWNXh+2nA/p2e8z3C0RAthB11EP5j6ac2k2YBt8YOUc1UuqX3XTZ2UTRrgEeBMzvd39UY7wLgpOT794GjgQOBw4CTgQkdnvufwKGEteEBwJHAAYRx4APTiy/lc2VySruUiMZ0y8HsXwhDniKV7He4d/69LClT6ZZDuMj5G2w6nCpSSdYC++I+J3aQaqfhhXIIFzm/JnYMkS34JxVueWhNt5zMHgOOjx1DpJNGYLSOWCgPremW15cIJ3CJVAoHLlThlo9Kt5zc3wGuix1DpIOf4f5Y7BC1RMML5WbWg3Dm7ZGxo0jNewM4GHdtfZWRSjcGs90IJ3LtEDuK1KxWwpTqf4kdpNZoeCEG9/eAi2LHkJp2owo3Dq3pxqSTJiSOF4AjdeZZHCrdmMx6A9MJky+IlMNi4FDc340dpFZpeCEm93WEC3ytjh1FakIr8GkVblwq3djc3yRcjlabHFJqV+P+ZOwQtU6lWwnc7ydcIVGkVO7C/Z9jhxCN6VYWszuBC2LHkKrzZ+DjyXCWRKbSrSRmPQkTPOj6DJKWBYQdZwtiB5FApVtpwmUgnwX2jR1FMm8ZYQ33ldhBZCON6VaacBnIkwiTNogUag1wigq38qh0K1E4Y+0ThGMqRfK1HjgL92djB5GPUulWKveZwHGE6bBFuqsV+CzuU2MHka6pdCtZ2DRU8Up3bQDOxf13sYPI5ql0K537q6h4ZevWA+ckx3xLBdPRC1lhth/wBDA4dhSpOKuBM3Qx8mzQmm5WuM8CxgNzIyeRytIMHKvCzQ6Vbpa4v0aYcWJG7ChSEd4Fjsb9hdhBpPtUulkTziw6BvhD7CgS1SvAUckFkyRDVLpZ5L4KOAX4aewoEsV/Acfo1N5sUulmlfsG3C8HLiEcmynVrw24nnCm2fLIWaRAOnqhGpj9DXAPMDx2FCmZpYRDwjSslHFa060gZnanmS0ys1fzeqH7fwMHAg+WJJjE9hIwVoVbHVS6leUuYEJBr3RfivvpwFeAtSlmkrjuJOwwezd2EEmHSreCuPufCJuRxSzkZ8DhwOtpZJJomggnPFyEu36JVhGVbjUKF8sZC/wrmnsti+4G9sP9P2MHkfSpdKuV+xrcLyEc0zsrdhzploXAabifi3txWzxSsVS61c79aeBg4FvAh5HTyOZNIazdPhQ7iJSWSrcWuLfg/kNgP8KB9VI5ZhKunfAF3JfFDiOlp9KtIGZ2L/AcMMrM5pnZRam+gfsc3E8GzgbeSXXZkq/FwKXAIbg/FTmLlJFOjqhVYebhC4HvopMqymkNMBm4CfeVscNI+al0a53ZNoRTib+FrtVbSuuBXwA36JoJtU2lK4HZdsAVwDXAgMhpqslK4DbgJ7gvjB1G4lPpyqbM+gIXAZcDIyOnybIm4CfAbbiviB1GKodKV7pm1gM4DfgqYcYK6Z7XgB8DU3BfFzuMVB6Vrmyd2T6Ecd8vAv0jp6lEy4H7gbtwnx47jFQ2la50X9jpNgH4DDAR2C5uoKg2AI8SLlL0oK6PIN2l0pXCmG0LnEwo4JOBbeMGKosW4Gng98B9OgpBCqHSleKZbQ+cBPwtcCywR9xAqVoITCWcyfeojq2VYql0JX1muxLK97jk6y5xA+VlGTAdeIZQti+jHxJJkUpXSs9sD8KlJg/ocBsJWMxYwGrCdPYvJbfpwJsqWSklla7EEYYk9icU8CjC2nD7bQjQM4V3ccLxsnMI15qY0+n7ebi3pfA+It2m0pXKY2aEs+IGAw2EnXS9klvPDt/3IsyQu4pw5tdHb+4t5Y4vsiUqXRGRMtKlHUVEykilKyJSRipdEZEyUumKiJSRSldEpIxUuiIiZaTSFREpI5WuiEgZqXRFRMpIpSsiUkYqXRGRMlLpioiUkUpXRKSMVLoiImWk0hURKSOVrohIGf0vrqH4lmgpU28AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# define the pie drawing function of probability analysis\n",
    "def plot_pie(prbs):\n",
    "    dict1 = {}\n",
    "    # remove the negative number and build the dictionary dict1. The key is the number and the value is the probability value\n",
    "    for i in range(10):\n",
    "        if prbs[i] > 0:\n",
    "            dict1[str(i)] = prbs[i]\n",
    "\n",
    "    label_list = dict1.keys()\n",
    "    size = dict1.values()\n",
    "    colors = [\"red\", \"green\", \"pink\", \"blue\", \"purple\", \"orange\", \"gray\"] \n",
    "    color = colors[: len(size)]\n",
    "    plt.pie(size, colors=color, labels=label_list, labeldistance=1.1, autopct=\"%1.1f%%\", shadow=False, startangle=90, pctdistance=0.6)\n",
    "    plt.axis(\"equal\")\n",
    "    plt.legend()\n",
    "    plt.title(\"Image classification\")\n",
    "    plt.show()\n",
    "    \n",
    "    \n",
    "for i in range(2):\n",
    "    print(\"Figure {} probability of corresponding numbers [0-9]:\\n\".format(i+1), prb[i])\n",
    "    plot_pie(prb[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上过程就是这次手写数字分类训练的全部体验过程。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
